If you’re like me, then you must have gone nuts on hearing about WebSockets for the first time. We now have a totally independent protocol that will not terminate connection after a request-response cycle, which means all hacks to keep a persistent connection from the browser (long polling, I’m looking at you) would now be part of the history. Welcome to the world of WebSockets.

Let’s start with the Wikipedian and Mozillian definition of WebSockets, to get some initial traction.

Wikipedia says that..

WebSocket is a protocol providing full-duplex communication channels over a single TCP connection. The WebSocket protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API in Web IDL is being standardized by the W3C.

Mozilla Developer Network says that..

WebSockets is an advanced technology that makes it possible to open an interactive communication session between the user’s browser and a server. With this API, you can send messages to a server and receive event-driven responses without having to poll the server for a reply.

To begin with, let us all make sure we have websocket support by clicking here. Assuming you do, let’s begin. Web browsers provide us with the WebSocket class. To create a websocket, we simply need to

let ws = new WebSocket('ws://server.com/endpoint');

This gives us a websocket object which supports methods like send() and close() and event listeners like onmessage and onerror.

Creating a nodejs instance running a websocket server is easy with modules. I did try to do it natively, but it went too tedious. Finally I settled for ws module from npm. The server side code looks trival.

On the browser, things are even simpler.

// call the object constructor
var ws = new WebSocket('ws://nagekar-ws.herokuapp.com/'
// attach a message listener
ws.onmessage = function(msg) {
	console.log(msg
}
// send some message
ws.send('hello server!'

Once we call the constructor, an HTTP GET request is generated to initiate the transaction. You’ll find that these headers are sent by your browser.

GET / HTTP/1.1
Host: nagekar-ws.herokuapp.com
Connection:Upgrade
Pragma:no-cache
Sec-WebSocket-Extensions:permessage-deflate; client_max_window_bits
Sec-WebSocket-Key:ut0NjBQxxBnmtUStHfMUDw==
Sec-WebSocket-Version:13
Upgrade:websocket
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.108 Safari/537.36

The ones to notice are the URL, which begins with ws:// protocol. Cross domain requests are allowed. This initial step requires the use of HTTP(S) or web, hence the name ‘web’ sockets. The ‘Connection:Upgrade’ and ‘Upgrade:websocket’ and what requests the server to change this connection to a socket one. The Sec-WebSocket-Key header is what client sends to the server. The server crafts another key with this and returns it back in the response header. A response header looks like this.

HTTP/1.1 101 Switching Protocols
Connection:Upgrade
Sec-Websocket-Accept:7CAJHdL7QMG4ceqd0M1O8SDbM2Q=
Sec-Websocket-Extensions:permessage-deflate
Upgrade:websocket
Via:1.1 vegur

On receiving the Sec-Websocket-Accept header, the handshake is completed, and now the client and server are connected via persistent sockets, ready to perform both way message delivery.

Click ‘Connect’ button to initiate WebSocket handshake. The server will respond with a timestamp every 2500 milliseconds. Also, the server will echo back everything to send at it. The close button will call the WebSocket.close() method which will terminate the connection.

Message Box

Pretty interesting, isn’t it. Things get even more interesting when you use a readymade library built on top of WebSockets, such as Socket.io. It supports all browsers (through fallback to XHR/polling), has better looking APIs, is easy to use and allows you to broadcast without having to worry much about raw connections. If you’re going to use these WebSockets in production, there’s no reason not to use Socket.io. Using native websockets is not recommended if you’re planning to support some legacy browsers. Even Mozilla terms it as an ‘experimental technology’. Check out the compatibility table for more information.

I hope this post was informative. I did learn a lot while researching on this topic, and I’m glad I did. As always, thank you for reading. Please drop a comment in case of feedback or correction.

I remember starting to learn Javascript. It was all a nightmare. Functions used to execute out of order and seemed very random. There was chaos all around, and I was usually left wondering, ‘what is the order of execution here?’. Of course, that question never got answered, because that wasn’t the right question. The right question would’ve been, what drives these functions in this code.

Fortunately, I asked it at an early stage. The things that make Javascript tick are called events. Without these, your browser window is just a dead canvas. Not just the browser, but other Javascript environments as well, work because of events. And when you understand it, events and event driven programming are probably the most intuitive form of developing applications. Events are not difficult to visualize, once you start to relate them to the real world events.

Suppose you’re alone at your home. You’ll need to make sure you open the door in case someone arrives. There are two ways of doing it. First way is to periodically keep checking if someone is outside the door, say every 5 minutes. In the best case, someone will arrive the moment you open the door, but in the worst case, a visitor has to wait for about 5 minutes before you make your next visit. This is just ridiculous. It is inefficient (visitors have to wait for you) and waste of time (when you check but no one has arrived). Who does this? Well, most procedural languages work this way. And it works, for their use cases. But when you’re dealing with real life scenarios, like checking doors or handling HTML form responses in browsers, where the time of completion of an action or arrival of a visitor is unknown, a better approach has to be used, especially when time, muscle and browser resources are so scarce. Here comes event driven programming.

Suppose, you’re alone again, in the same house. But this time, you decide to make use of events. You install a doorbell. Now, whenever someone arrives and rings the bell, you know you have to go open the door. That’s simple. That’s event driven. It doesn’t matter if the next visitor comes after a minute or after an hour. You’ll open the door instantaneously, without waste of your own time. And the fact that most homes have doorbells and this is our natural workflow when dealing with similar situations, is the proof of how intuitive this programming paradigm is.

Okay, that was cool, so far. But I know that I have to listen to a doorbell and then on hearing it, walk to the door to open it. How do I make the machine do that. The answer is callbacks. If you are not quite sure what they are or how they work, do check out my other article on how callbacks and event loop work. So essentially, we attach a ‘callback’ to an event. The callback, as the name suggests, calls back when the event occurs, and lets us control the aftereffect. Such a callback is also called an event listener, which infact is what you attach to the element.addEventListener('event', callback); when working in the DOM. In Nodejs environment, you get a custom EventEmitter class that you can use to emit and listen to events. If you haven’t heard of it, socket.io is all about custom events and the data transfer between the client and the server. It makes use of websockets along with XHRs. If you are into creating sockets and data interexchange between remote hosts like I am, socket.io is for you.

Finally, I hope to have given you some insights of event driven programming with Javascript. Not a lot of code, but that is something you’ll find everywhere. I always like to emphasis on the concepts rather than implementation. It makes for a better implementation.

I have been a part of numerous web projects, and after these many applications using multiple languages and frameworks, when my friend Kunal suggested that we should develop an Android application that would make use of a backend and stuff like that, I couldn’t help but desperately show my excitement. I told my friends, family and even seniors about the thing I am working on. After all, it was my first time going full stack mobile.

For clarifying things, I am still doing the backend. It is my little team that has gone full stack. On mobile. The application, called the moodShare will enable a user to do few simple tasks like registering, logging in, adding friends, sharing statuses, reading statuses and stuff like that. Each task is made possible with a simple API that is made without much thought to future scaling. I know, that is really bad from my side. But then, I was too desperate to get an application up and running that I neglected a lot of issues that arose when we were developing.

The application’s backend was Mongo, Express and Node. We deployed it to Modulus.io but then this happened and we had to shift it to Heroku.

Too poor to get a proper server… Crashing in 3..2..1 #javascript #nodejs

A photo posted by Abhishek Nagekar (@abhisheknagekar) on Mar 4, 2016 at 9:06am PST

And this was when we had around 4 live users. The code quality was poor, very poor. Trust me when I say it. 4 users were hitting the database around 7 times a second.

Beautiful sight of requests hitting the API server #javascript #nodejs

A video posted by Abhishek Nagekar (@abhisheknagekar) on Mar 4, 2016 at 7:43am PST

Then there were many other issues as well. For example, if you added a friend who had 400 status updates in his or her name before you added, your phone would notify you, well, 400 times. Literally. That was awful. Push notification was implemented in a way you write an infinite for loop to check if something has changed.

But hey, this was our first time. We did enjoy each and every second of the development (around 4 days), the bugs just as much. We did learn a lot from it. Before this project, messaging queue was just a name, Google Cloud Messaging was just another service that I knew nothing about, three or four HTTP status codes that I knew. The satisfaction of knowing more about these was worth it all. Lastly, we did not abandon the project. Our alpha version was pretty stiff and unusable. Hopefully, with better APIs, better hosting and material design on the frontend, we plan to rock with the beta version. Stay tuned.

Screenshots

Links

• The initail release of moodShare
• Backend code on Github
• Live API on Heroku

Callbacks are the things that make writing Javascript fun. If you think you are better off writing Javascript the way you wrote C and Java, then you are not really feeling the essence of this beautiful language. I found it rather hard to get used to this style of programming, called affectionately as the non-blocking style. What is this difficult sounding technique, and how can you make use of it to write better code is what we’re going to see in this little essay.

Starting with the most basic explanation of what a callback is. Suppose I have two functions A and B. Although I can execute both of them myself, what I can do is execute A and ask A to execute B for me. Yes, that is all there is to the callbacks thing. I have two functions, I execute one and then ask it to execute the other one for me. Simple.

But, you might ask, why I would ever want to do such a thing. Well, think about it this way. Computer processing is way, way faster than any I/O in question, especially when the I/O is on the other side of the network (or globe, for that matter) which is usual in the case of most web applications. Now, if you write code such that it queries a remote (or local) database and uses those results to carry on further processing, it results in a bottleneck. The processor executes instructions in the order of nanoseconds. An I/O request to the disk or network takes a few milliseconds at best, or a million times more time than what a processor needs to do a single atomic task. So essentially writing code that waits for results from I/O is simply wastage of the precious CPU cycles that could’ve been put to better use in the meantime.

A callback is also of great use when you want to make a function, say A, do things according to the context by supplying a specialized function B at the runtime. For example,

So let’s get into some (pseudo) code. We will first see what it is like to write code without callbacks and blocking IO and then examine some issues. Then we will write the same code using callbacks and non-blocking IO and see if we have rectified (or at least mitigated) those issues.

Now this code is blocked when the Javascript engine is waiting for the Mongodb query to execute and return. It may take a few milliseconds, and only after that does the control of the program moves to the next lines. If the database hangs up for any reason, then the response will never reach the end user, regardless of whether the login credentials were correct or not (or let the user know something went wrong). It may also happen that the Javascript engine continues the execution with ‘undefined’ returned, resulting in falsy block being executed each time. Having addressed these problems, let us now write the asynchronous version of the same (pseudo) code.

The callback function in a way keeps waiting for the response from the database in the background white the flow of program is not interrupted. As soon as the reply comes, the code in the callback function is executed. That sounds neat. Wait a second. Didn’t we all learn that Javascript is single threaded and does a single job at a time and what not? Then how is the callback supposed to listen when the browser is already executing the code below it?

To understand that, you’ll have to stop believing every thing you write in Javascript is Javascript. Yes, I mean it. Javascript is single threaded, but there is much more to Javascript than just a single call stack. Ok first, what is a call stack. Whenever a javascript file is executed, the engine creates a context for the code to run. This is the global execution context. It sits at the bottom of the call stack. Whenever a new function is invoked (or ‘called’), a new execution context is pushed on top of the stack. After the execution completes, the execution context is popped off the stack. After the last line of code is interpreted, the global context is popped off the stack.

Well and good. But what about that setTimeout function that you set for the next 10 seconds? And our own mongodb query. Where did those go? The setTimeout is actually a webAPI provided to us, the Javascript developers, by the browser. Similarly, our mongodb query command was not a Javascript thingy, but a C++ API from the Nodejs bag of goodies. These events are handled by their respective environment, and when they finish execution, they enter what is called the callback queue. There, they wait until the call stack is completely empty. Once the call stack in empty (all Javascript code is done running), the event loop kicks in, pushes the callback functions waiting in the call back queue on top of the stack, one by one. A great way to visualize what the above paragraph just said is to try it out yourself at loupe by Philip Roberts. He’s an amazing guy and you should totally check out his talks.

So that was it for this little article. Hope to have helped you. Keep digging.

Edit: The section below this has some major issues, that I only found out after pushing into github. Read edit section.

I always admired the sites which did not reload upon clicking links here and there, but simple gave me whatever I requested for, with grace. Sites like Facebook, Twitter and Youtube extensively use it, and the application hardly reloads after the initial reload. Here’s my attempt on making something similar.

Firstly, I downloaded a free HTML template from Online Casinos. The links had to be edited. The pattern I used was to replace, for example, any (internal) link pointing to example.html to #/example.html. What it did was 1.) prevent the page from reloading, and 2.) putting the requested location to some place from where I can extract it with Javascript.

Next, grab that location of the hash. The Javascript’s built in method window.location.hash provides the page’s current hash. We drop the first char from the returned hash as it is the hash (#) character itself. The hash value is essentially the location to be navigated to, hence, we use some string manipulation to craft a new URL from it, and make an AJAX request to fetch the contents of the page requested, and insert it into the div specified through Javascript.

For example, suppose we are on about.html page. Now if I click ‘contact’ link, the link would change to /about.html#/contact.html, the hash value would be grabbed by the window.location.hash method, a GET request is fired to get that location with a XMLHttpRequest object, the response is inserted with innerHTML method on the div, and using the window.history.pushState() method, we update the URL to /contact.html, to ease sharing. All this, in the blink of an eye. Cool.

I could do it in two different ways. One was to just place the text in those files and inserting them into the template. The drawback of this was that if a user directly visits that page, he/she would just see plain text. For example, check this link out. The Javascript code for that was,

The separate callback function for getHash was necessary because onclick() event fires before the hash is changed, and it would result in the script grabbing the previous hash. As already mentioned, the navigation is smooth if one visits index.html and then other links, but try getting the about.html page first. All you’ll see are a few lines of text.

A sloppy solution to the above problem was to replicate the entire index page with varying content for each of the page, and then replace entire DOM on anchor click. You can imagine this is not at all smart, and would destroy the point of this site. Not to mention the extra bandwidth to reload the same resources on every request. Here is the implementation for this method.

What would be the smarter way? Perhaps, making sure the first page to load always downloads the entire html every time, and subsequent requests to be made to download the changed content. I will be implementing this solution shortly. Drop in your ideas too, and share if you enjoyed reading this.

Edit

So, the above code seemed to work on my localhost, but there were some serious (cross origins and some logical ones) issues with it, so much that I considered to learn relative and absolute links all over again. Anyways, I also happened to redesign my blog during this time (redesign in the sense changing the template), and I am working to implement AJAX loading on all links here on my blog. So far, all links on the home page seemed to work as expected (AJAXified). I am using the document.write() method, so judge me on that. You can check them out here. The code snippet is following.

What do you think about it? It feels a little buggy at times, and sadly, Internet Explorer isn’t supported yet. But for most of it, chrome and firefox are working like a charm. Hope to get all links covered under this thing, which would be nice. Sadly though, the browser tab shows loading for all the third party resources on the page which kill the purpose of this thing. Anyways, feel free to correct or improve this in the comments below.