Come hear my rant @ Göteborg on the topic of #GDPR

You know you want to 🙂 Come hang for the rant, but stay for MĂĄrten RĂĄnge‘s session on Property Based Testing.

update 13.04.2017: slide deck avail here: https://github.com/yveshwang/presentations/tree/sthlm/impress.js/gbg

Implementing GDPR & Property Based Testing

Wednesday, Apr 12, 2017, 5:00 PM

Atomize AB
Västra Hamngatan 11 Göteborg, SE

25 Members Attending

Onsdagen den 12:e april kl17:00 välkomnar Atomize AB tillsammans med SWENUG er till ett event där Yves Hwang frĂĄn Circle K kommer prata om hur han implementerade GDPR där. GDPR kommer 2018 och ersätter PUL och berör alla företag och myndigheter som handskas med känsligt data som berör EU medborgare. Yves bor i Norge men härstammar frĂĄn Australien d…

Check out this Meetup →

Websocket: the road to hell is paved with good intentions

“Server-push technology” was an elusive sales pitch half a decade ago. COMET, Flash and websockets promised ways for servers to “push” messages onto clients. We sold our souls to heavy weight libraries like Icefaces, GWT, or some other JSF abomination that included hixie-75/hixie-76 support. We became oblivious to this devil’s promise and started blaming browsers when stuff did not work. Now with RFC 6455, the realities of websocket for me is still as disjointed as ever. I started off as an evangelist and a staunch supporter for websocket, but as Captain Hindsight would say, “you are doing it wrong”, and I quickly learnt from my mistakes.

This blog is my personal journey with websocket. In the era of devops and cloud (just as elusive and sales pitchy as ever), in my opinion, I find it really hard to see how this protocol would fit elegantly into an elastic, edge-side, micro-service cloud architecture/infrastructure.

tldr;

Websocket does not work out of the box for most load balancers. Furthermore, when you upgrade the connection directly to the backend (see this piece I wrote for Varnish a while back) you lose edge-side load balancing and caching, thus essentially piping the backend directly to the browser one connection per client at the time. Without some additional clever connection multiplexing components in between the load balancer and the websocket server, like websocket-multiplex or some tricked out node-http-proxy, the websocket server will not scale. For those that prefer sales and marketing lingo, this means it is not “cloud enabled” or “web scale.” Furthermore, websocket’s binary protocol, implemented by libraries such as jWebsocket, is extremely hard to debug in production. Unless you are super savvy with Wireshark and regularly eat TCP dumps for breakfast, and not to mention a bit of a masochist, I highly recommend staying away from websocket all together at the time of writing.

Websocket in practice, and asking some hard devops questions

In the past, I have had the displeasure of working with Icefaces and GWT. These Java front end frameworks abstract away the nitty gritty of the network and protocols such as websocket versions, handshakes, messaging format and error handling with elegant MVC models. This is all well and good on the drawing board, but MVC is a design pattern for separation of concerns on the UI level. Not exactly applicable when talking about the complexity and reality of running a websocket server in this internet and mobile driven world.

I have spent past 4 years developing and supporting an application that first utilised websocket. I have to admit, it was fun building directly ontop of jWebsocket, but it was painful and nearly impossible to debug in production. This was alleviated when we went full blown backbone.js, require.js and d3.js. Keeping things simple pays off in the long run. From that experience, I have devised a checklist for any future websocket projects, and help potentially avoiding the same situation from happening again.

  • Are there any authentication with this websocket implementation? If so, how does this tie into the user model? (jWebsocket for example requires you to specify a couple of system users for authentication and namespaces/roles for authorisation. These are separated from the existing authentication/authorisation model used by the webapp)
  • If it runs within browser, can you tweak the client-side retry logic or connection timeout variables?
  • If this runs outside of a browser (and this gets weird fast) are there any TCP client that can be used for automated testing or health checks in production?
  • How do you test this automatically? Is it supported by Selenium or phantom.js?
  • Can this webserver server be plugged into existing load-balancer? Any additional tweaks and settings required?
  • Does this need multiplexing?
  • How do you debug this on client-side in production? This is usually not possible because the connection is now elevated into a TCP connection and browsers no longer cares for it.
  • How do you debug this on server-side in production? This gets even more tricky as you include multiplexers and load balancers and various other nodes that speaks this binary protocol.
  • How do you debug this at all? Not possible if browser gives up the connection and everything in between is a mystery.
  • Ok so we can do TCP dump and Wireshark. So are you ready to do TCP dump between somebody’s browser and the origin server and everything else in between?
  • Catering to older browsers means Flash fallback. Are you prepared to open an additional Flash port and start supporting this? (and repeat the set of debug and test questions for Flash fallback)
  • Does this thing scale? Yes if you multiplex the connection.
  • How does it handle reconnection and stateful connection?
  • How does the server side handle connections and thread pools handling these logics behind each connection?
  • Does the network infrastructure block upgrade headers? Any other firewall rules that might break websocket?
  • Lastly, you must be prepared to give up caching.

Benefits with websocket

  • True server-push! Yes, this is true. Long-polling or COMET is not exactly “pushing” messages per definition. You are safe from the definition police.
  • You get to play with fun new websocket multiplexing code. This is quite cool actually. Mucking around with node.js is always fun. Perhaps you are thinking about building a Varnish VMOD to support websocket multiplexing. Perhaps you are thinking about building some kind of HTTP cacheable stream for websocket messages before stitching them back out as websocket payloads? This is all very exciting indeed!
  • Ability to build cool HipChat-like applications

#leetspeak ’14 for me: big ball of mud, devops, architectural validation

Nothing in I.T. is new. We merely learn to call it different names. We continue to battle with big ball of mud. As this guerilla warfare thickens, we learn to scope down and manage smaller balls of mud through tighter collaboration and empathy between dev and ops. Devops anyone? The war against object-oriented programming rages on, backed by experts and Hacker News alike. It seems that we need an escape goat but even as objects are denounced as the new evil one thing remains constant, Python is a great language with a shit runtime and Java is a shit language with a great runtime. Oh and there is still no love for Visual Basic or Sharepoint developers. Actors, messages and events are the new hipster but have they not been around since Erlang or Cobol? We give it a new name, a new stage like C10K and the internet and mobility, forgetting the lessons of old. Nothing in I.T. is new.

It is not often that one gets a total validation to previous architectural design and implementation by the likes of Spotify. I have long defended and championed my Java expertise in the land of sysadmins dinosaurs, where empathy seems to be part of a bike shedding exercise. From this baptism of fire I realised it is not the language, it is the approach and architecture design that matters. A choice we make that will determine the construct and the size of this ball of mud. I walked out of Niklas Gustavsson’s talk thrilled, overwhelmed with joy and kinship. For I am not alone. For the stuff I architected in the past are composed of the same technical choices, principles and components. A big pat on my own back, and we need to do that from time to time 🙂

Then this brings us to devops, collaboration and throwaway code. Well, if Gartner said that it is here in 2015, then the pitch is even easier to convince the execs.

Devop is not a position, title or an engineering degree. I have written in the past that devops is culture. It is empathy and collaboration. For us grunts and peons navigating the corporate waters, this is the ammunition we need to break down silos, poor communications, ridding ourselves of these monolithic non-scalable balls of muds. This is a grass root movement that will change the faces of I.T. departments worldwide.

Indeed it is a rarer occasion to be surrounded by software craftsman of similar background that are driven, passionate and willing to share their experiences. For that I am grateful to both Tretton37 and Gothenburg. If you have missed leetspeak 2014, fear not, it was recorded. So thank you again and I will definitely be back for next year.

Now time to assemble my devop Avengers. We got work to do.