All posts by Abhishek Nagekar

The Pragmatic Programmer Notable Quotes

Uncategorized,

I completed reading ‘The Pragmatic Programmer’, probably one of the most recommended pieces of literature on software engineering. I’ll try to put down some of the quotes and my interpretation of them in the following paragraphs. I’ll be coming back to this from time to time. Hope they help you in your next software project.

The Cat Ate My Source Code

The greatest of all weaknesses is the fear of appearing weak.

Take responsibility for the code that you write and the decisions that you make, and when things go south, present options to recover from the mess instead of pointing fingers.

Software Entropy

When disorder increases in software, programmers call it “software rot.”

Active efforts have to be made to keep the disorder low and go against the nature’s law (of ever-increasing entropy) to keep away from software rot.

Don’t Live with Broken Windows

Once there is a broken window in a building, there will be more broken windows. This is the broken windows theory. Similarly, once a piece of sloppy code enters a codebase, it will attract more sloppiness. Don’t be the one to introduce a broken window into your codebase and fix existing ones as soon as feasible.

Stone Soup and Boiled Frogs

The frog just doesn’t notice the change. Don’t be like the frog. Keep an eye on the big picture. Constantly review what’s happening around you, not just what you personally are doing.

If you throw a frog into boiling water, it will jump out at your face. But if you put the frog in cold water and boil it, it won’t notice and get cooked to death. Good software projects die just like that, slowly, without anyone noticing.

Good-Enough Software

But artists will tell you that all the hard work is ruined if you don’t know when to stop. If you add layer upon layer, detail over detail, the painting becomes lost in the paint. Don’t spoil a perfectly good program by overembellishment and over-refinement.

Know when to stop refining a piece of software. Make tradeoffs if they improve the overall quality of your software.

Your Knowledge Portfolio

  • Serious investors invest regularly—as a habit.
  • Diversification is the key to long-term success.
  • Smart investors balance their portfolios between conservative and high-risk, high-reward investments.
  • Investors try to buy low and sell high for maximum return.
  • Portfolios should be reviewed and rebalanced periodically.

Think about your knowledge portfolio as investors think about their investment portfolio.

Communicate!

It’s not just what you’ve got, but also how you package it. Having the best ideas, the finest code, or the most pragmatic thinking is ultimately sterile unless you can communicate with other people. A good idea is an orphan without effective communication.

Learn to communicate with different kinds of people, technical and non-technical alike.

The Evils of Duplication

DRY (Don’t Repeat Yourself) principle: EVERY PIECE OF KNOWLEDGE MUST HAVE A SINGLE, UNAMBIGUOUS, AUTHORITATIVE REPRESENTATION WITHIN A SYSTEM.

Documents are no exception. Don’t duplicate business logic in code AND documentation. There should be one true source of knowledge (model) and the others should simply be views of that source.

Orthogonality

In computing, the term has come to signify a kind of independence or decoupling. Two or more things are orthogonal if changes in one do not affect any of the others.

The code you write has to be as decoupled from the rest of the project as possible. It makes bugs easier to find. There should never be overlapping functionality or duplicate code (also a violation of the DRY principle).

The Basic Tools

Every craftsman starts his or her journey with a basic set of good-quality tools.

Selecting a good set of tools can greatly improve your productivity and skills.

Source Code Control

Always Use Source Code Control

Even on single person projects that’s never going to see the light of the sun.

Debugging

Embrace the fact that debugging is just problem solving, and attack it as such.

The best way to start fixing a bug is to make it reproducible. After all, if you can’t reproduce it, how will you know if it is ever fixed?

“select” Isn’t Broken

If you encounter a bug, always be suspicious of your own code. No, the compiler isn’t broken and nor is the CPU. It is your code, fix it.

Pragmatic Paranoia

You Can’t Write Perfect Software

But Pragmatic Programmers take this a step further. They don’t trust themselves, either.

When everybody actually is out to get you, paranoia is just good thinking.

We overestimate our capabilities often. Most people think they’re the best drivers out there. Pragmatic programmers understand that everyone, including themselves, make mistakes, and hence keep an eye out for bugs before they creep in.

Dead Programs Tell No Lies

When your code discovers that something that was supposed to be impossible just happened, your program is no longer viable. Anything it does from this point forward becomes suspect, so terminate it as soon as possible. A dead program normally does a lot less damage than a crippled one.

Assertive Programming

If It Can’t Happen, Use Assertions to Ensure That It Won’t

Turning off assertions when you deliver a program to production is like crossing a high wire without a net because you once made it across in practice. There’s dramatic value, but it’s hard to get life insurance.

Always use assertions to make sure things are working well, even in situations when you’re sure or things are ‘obviously simple’. Detect early and abort.

When to Use Exceptions

“Will this code still run if I remove all the exception handlers?” If the answer is “no,” then maybe exceptions are being used in nonexceptional circumstances.

Don’t use exception for normal code flow control. Exceptions need to be reserved for ‘exceptional’ situations.

Decoupling and the Law of Demeter

Organize your code into cells (modules) and limit the interaction between them. If one module then gets compromised and has to be replaced, the other modules should be able to carry on.

Metaprogramming

Configure, Don’t Integrate

Our goal is to think declaratively (specifying what is to be done, not how) and create highly dynamic and adaptable programs.

Program for the general case, and put the specifics somewhere else—outside the compiled code base.

Put Abstractions in Code, Details in Metadata

Because business policy and rules are more likely to change than any other aspect of the project, it makes sense to maintain them in a very flexible format.

It’s Just a View

The view is an interpretation of the model (perhaps a subset)—it doesn’t need to be graphical.

The controller is more of a coordination mechanism, and doesn’t have to be related to any sort of input device.

Understand what a view really is. Everything revolves around the models, and views are just different ways in which you can represent those models.

While You Are Coding

However, good, safe drivers are constantly reviewing the situation, checking for potential problems, and putting themselves into good positions in case the unexpected happens. The same is true of coding—it may be largely routine, but keeping your wits about you could well prevent a disaster.

Programming by Coincidence

Fred doesn’t know why the code is failing because he didn’t know why it worked in the first place.

If you don’t have fundamentals or infrastructure correct, brilliant bells and whistles will be irrelevant.

Program consciously. Write code thinking of it as an autonomous task once you have the business requirements and architecture clear is a mistake. There has to be equal involvement during programming.

Refactoring

Rather than construction, software is more like gardening—it is more organic than concrete. You plant many things in a garden according to an initial plan and conditions. Some thrive, others are destined to end up as compost. You may move plantings relative to each other to take advantage of the interplay of light and shadow, wind and rain. Overgrown plants get split or pruned, and colors that clash may get moved to more aesthetically pleasing locations. You pull weeds, and you fertilize plantings that are in need of some extra help. You constantly monitor the health of the garden, and make adjustments (to the soil, the plants, the layout) as needed.

If it hurts now, but is going to hurt even more later, you might as well get it over with.

Code That’s Easy to Test

All software you write will be tested—if not by you and your team, then by the eventual users—so you might as well plan on testing

The Requirements Pit

Don’t Gather Requirements—Dig for Them

Work with a User to Think Like a User

The key to managing growth of requirements is to point out each new feature’s impact on the schedule to the project sponsors.

Ubiquitous Automation

Civilization advances by extending the number of important operations we can perform without thinking.

Automate as much as you can.

Ruthless Testing

Most developers hate testing. They tend to test gently, subconsciously knowing where the code will break and avoiding the weak spots. Pragmatic Programmers are different. We are driven to find our bugs now, so we don’t have to endure the shame of others finding our bugs later.

Test State Coverage, Not Code Coverage

It’s All Writing

One of the most important pieces of information that should appear in the source file is the author’s name—not necessarily who edited the file last, but the owner.

Pride and Prejudice

We want to see pride of ownership. “I wrote this, and I stand behind my work.” Your signature should come to be recognized as an indicator of quality. People should see your name on a piece of code and expect it to be solid, well written, tested, and documented. A really professional job. Written by a real professional. A Pragmatic Programmer.

Take pride in your work. Don’t just assume it will be a small cog in the big wheel and use that as an excuse to write sloppy code.

In Closing

That’s all. There’s a lot more in the book, the stories and the experiences. I highly recommend you give it a try if your day job involves writing computer code. Thanks for reading.

Testing Websites On Kindle Paperwhite

Uncategorized,

I bought an Amazon Kindle this week, a device for reading books. It does a magnificent job at improving your reading speed, giving meanings and Wikipedia lookups on the spot and in general making reading a much more convenient and peaceful experience (for a non-native reader, at least).

I was pleasantly surprised to find an experimental web browser in it. I wanted to know how experimental it is. This post is going to be about that.

Warning: Multiple images ahead. Might take a few seconds to load. Patience.

Browser Score

One way to check how good or modern a browser is, is by checking and comparing html5test.com. While not all the features are made equal (some might be more important than others depending on the situation), it definitely gives us a good ballpark figure to compare browsers. For example, Latest Firefox (62.0.3) scores 493 on that test, while lastest Chrome (69) scores 505. Let’s see how our Kindle browser performs in this test!

Hmm. Not bad. And given that it supports Javascript (which was a little surprising to me), I think it works pretty well for its ‘experimental’ tag. Not web 2.0 of course, but decently well.

Testing Websites

Benchmarks and synthetic tests do their job, but nothing beats real-world web browsing, right? I tried to spend some time surfing the web with the Kindle’s browser to see if it would work for casual browsing, or any browsing for that matter. Let’s see how some of the popular websites performed on the browser.

Google

Google was pretty functional, and I didn’t have much trouble searching and navigating through search results. It felt very similar to how surfing on Opera Mini felt (remember that browser?). Usable, I’d say.

Gmail

In all seriousness, I didn’t expect Gmail to work. But it does and gives you a nice mobile view where you can read (and even compose) emails. Not the most convenient way of doing it (and using a randomized twenty-six character password didn’t help either), but can be done nevertheless.

Amazon

Amazon.com was kind of disappointing. Given how their shopping site looks even on modern browsers, I had expected it to look more or less the same, but the website essentially disappeared on Kindle.

The item listing page was a little better, although not a lot.

Wikipedia

Wikipedia was more or less what you’d see on a full browser, at least for reading existing articles. I didn’t try the contributors section (I’m lazy that way).

Youtube

The mobile Youtube site opens up with large thumbnails. On clicking a video, the media file (video.3gp as can be seen in the address bar) opens and as expected, doesn’t play. So Youtube is pretty unusable on Kindle.

xkcd

I couldn’t tell a difference between this version and the one I usually open on Chrome or Firefox. Nearly perfect. Nice work, Randall!

This blog!

Okay, I know. Not a popular website. One of the primary design goals while making the template for this blog was accessibility. I was very pleased to see everything in place (except for a few flexbox dependent sections on the homepage which needs a fix). Well done me! (I’ll show myself out)

Conclusion

I think Kindle’s little browser serves its purpose and does it well. Text is sometimes hard to read given the gray and black display. In my opinion, the bigger problem was that of scroll and typing. E-ink displays are very slow at refreshing, which makes scrolling web pages a pain. Then the typing experience is not that great either. I’m not complaining, of course. That would be very stupid. Honestly, I’m quite impressed with this device. It is well built and makes reading very convenient.

I want to think that the browser is so bare because the engineers at Amazon didn’t want us to switch to surfing Youtube or Facebook while reading that book. So in that sense, it isn’t a bug, it is a feature. Anyway, I hope you enjoyed this little article. I had nothing better to do on a Saturday evening, so this was it. Thank you for reading.

ELI5 – Warranty Canary

Uncategorized

I thought this would be interesting to share. I’m not aware of the legalities of this in countries other than the US and will need to confirm the same with my lawyer friends. Nevertheless, I found it to be very clever when I first encountered it.

So canaries are birds that coal mine workers used to carry with them to work. In coal mines, there’s always a danger of increased levels of hazardous gases. The canaries served as a warning system for the mine workers. When concentration of hazardous gases increased beyond what’s normal, the canary would die, warning the mine workers to leave before it gets dangerous for human beings (and hence the phrase canary in a coal mine).

A warranty canary serves a similar purpose for users of a website. In US (unsure about other nations), the governments and national security agencies can secretly order any organization to hand over user/customer data, and along with that order comes a gag prohibiting that company from discussing the order with anyone (its users, for example). Now internet companies needed a way to still circumvent this gag order (and maintain trust with their users, especially for companies like VPN and mail providers where trust is critical), and that’s where the canary comes into picture.

Warranty Canary

Librem Hardware warranty canary

The ‘warranty canary’ is a line of text mentioned on the website stating something similar to this:

Until now we have not received any letters, gag orders or warrants from any government

That’s it. This line says that the organization has never received any of the said requests. Once they do get such a request, they have to adhere to it. When they do, they remove the canary, hinting to their customers/users that their systems have been compromised by some government body but not violating the gag order that might be associated with the request. Clever!

For examples, see NordVPN or Cloudflare. For a list of major companies with warranty canaries, see the Wikipedia page. It can be seen that Apple and Reddit removed their warranty canaries which may indicate that they had been contacted by intelligence agencies.

That’s it for this little ELI5. Thank you for reading.

Accessibility-First Web Development

Uncategorized

When you zoom out a bit and think fundamentally, the web is just an input/output device for the end user. One can send and receive data and…umm? and that’s basically all you can do over the internet. When you think about it this way, the web seems like a lab experiment that got out of hand.

Now when I, as a web engineer, think of what the web is, I find myself in midst of complex jargon, standards, protocols, incompatibility and all the bloat surrounding web development these days. But for the end user, it is simply an IO device and that is something we mustn’t forget.

https://www.pexels.com/photo/laptop-technology-ipad-tablet-35550/

So if the web is fundamentally an IO device, shouldn’t we spend some time figuring out how to engineer things on top of it such that they are accessible to as many people as possible. I think so, and that is what I’ve been learning about these days. Without a doubt, the entire software ecosystem is getting more and more complex by the second, the merits and demerits of which are arguable. It gives me immense joy to work on SPAs and PWAs as an engineer. But not every website has to be an SPA. Heck, not every website has to have AJAX or even jQuery. Using plain old <form action=""> does not make you ancient. It makes you more accessible to people who have Javascript disabled.

But don’t stop there. Implement AJAX. Think about your use case and if it makes your or your visitors’ life easy, go ahead and drop React or any other modern web framework. But do so progressively, making sure your webapp is ‘functional’ for someone on a 320px wide browser on a slow internet connection as well as for any visitor on the latest iPad and 4G.

Let’s try to list down some tools and tricks we can keep in mind while developing our next web application to ensure that our web stays the medium it was meant to be; a library open to all, and not a shopping mall for the rich.

Progressive Enhancement

Progressive Enhancement is when you start developing a website with focus on core functionality and content accessible to as many users as possible, and then progressively keep on adding features for the modern browsers. That way, you satisfy the requirements and keep your site accessible. I suppose the only downside to this is that it will take more time to develop and build a web app this way, and I agree that time is usually the only commodity that startups and mid-sized companies don’t have these days. But most websites are not startup products, they’re information portals, personal blogs and wikis. No, I’m not discouraging startups from attempting progressive enhancement, but having worked at a lean-ish startup, I understand that this approach is not for everyone.

Speed

Shameless self-promotion

Your website has to be fast. It has been repeated by many industry veterans so you don’t have to believe me. We need to take pride not just in the way our websites look, but how quickly it loads. After all, we’re engineers.

One approach to this is to understand exactly what is being made. A personal blog does not need analytics code. Static social buttons with hardcoded links are much faster (and better from your user’s privacy perspective) than those dynamic ones. Use a CDN and HTTPS certificate, both of which are free.

Mobile Responsiveness

It really does not take much effort to make a website mobile responsive these days. Native CSS gives you enough tools to create nice grid layouts which respond to the different browser widths. I like to test my websites to a minimum of 320px, and have fixed max-width viewport (purely personal preference). Given the vastness of screen sizes these days, it helps me have a reasonable window for deciding how to collapse sections based on different widths. Responsive web design isn’t easy, but not very difficult either. Recommended reading: Responsive Web Design – Ethan Marcotte

Content

A major news outlet’s homepage – how not to do it

The content is the heart of your website, and it needs to be treated as such. Check out any popular Indian news website for guidelines on how not to place ads and prioritize content. If your website doesn’t resemble that level of junk overload for the user, you’re doing better than most.

Text & Colors

Text in lighter shades of gray on even lighter gray background look softer to eyes, but care must be taken not to overdo it. Remember that not everyone has a Retina or IPS display that you’re using to develop your beautiful webapp. It happened with my in LaughGuru. At some places, the text on the design mockups was so contrast-less, I had to change the angle of my monitor just to be able to read it (I had an old 1024×1280 square TFT LCD display on my desk).

In Closing

It is always nice to learn something new. I hope this learning continues, and I also hope you found some value in this article. Thank you for reading.

On The Importance Of Iterations In (Product) Development

Uncategorized

In LaughGuru, we received designs from the UI designers on Zeplin, a nice tool that bridges the gap between designers and developers. While I wouldn’t know about designers, it definitely made my life (as a developer) easy. My previous experience with frontend was with designers who gave me Photoshop PSDs and that was it. A Photoshop noob was expected to extract layers among other things and think about mobile responsiveness and all of that. I’ll leave you here saying it didn’t end well. But I digress.

Recently, I read the all-time classic ‘The Mythical Man Month’. It had an interesting line that I even wrote in my nice quotes file. It goes like this: “Plan to throw one away; you will, anyhow.”. It says something about planning to throw away the initial iteration of any product that you’re developing, because even if you don’t, you’ll throw it regardless. While that isn’t very agile and might not apply in its entirety to most of the software development happening today (to be fair, it was initially written in 1975), it had an interesting idea about how the first version of the product needs major reforms (or rewrite) to end up as something usable and/or beautiful. You can’t avoid that. You can only prepare for that.

Coming back to why I mentioned LaughGuru because I just remembered that. The first version that we received from the designers wasn’t very impressive. It lacked colors and looked bland to my ‘taste’, whatever it was at that time. But my manager had something else on his mind. He seemed to focus on the data flow and UX parts of things rather than the colors, unlike me. We went ahead and implemented it anyway, but since there was no design-design, most of the focus was on functionality, routing and data flow. It became the beta of our product’s latest iteration.

Later, we received newer designs which had colors. We discussed and later implemented them. This time, I could fully focus on the designs part as functionality was already done and tested. We did decently well and a couple of iterations later the end result was nothing short of beautiful. I used to tell my manager Look! it has started to look even better now, and he usually replied, unimpressed and with the typical attitude of a badass that he was, Yes, that’s kinda the idea. Of course, separation of concerns works. I was proud, and an important lesson got reinforced within me. The first iteration is usually ugly and sometimes embarrassingly silly. But it tells nothing about the practicality or viability of the underlying idea. It tells even less about if the idea will ever be successful. So why abandon it or feel dejected if the initial iteration draws negative feedback. Instead, learn from the criticisms and stick to the larger goal of why you started it in the first place.

I was recently working on my portfolio site because, to be honest, having this blog as a site to represent me and the kind of work that I do doesn’t really work with anyone expect developers and engineers, and that isn’t a very nice (read: accessible) thing to say about a website or yourself. I wanted something fancy and shiny, so I made that. Following are some don’t dos in any software project. I tried to focus on too many at once (content, UX, UI). I tried to get everything straight in one iteration, or roughly a day. I had no plan for the development, or any vision about what the end result should look like. I had little idea about what audience I’m targeting. The end result was something I wasn’t very proud, but I still asked my friends for feedback.

While the feedback was in general useful, a friend who was particularly into UX and UIs decided to guide me a bit (I’m sure he must’ve had taken a deep breath on seeing my work). He completed a course on product design on Udacity which mentioned that UI designing is never complete. You will always find ways to improve, and at a point, you’ll have to stop yourself once the design requirements are settled. But the room for improvements is always there. Another useful lesson. We went on to make small changes in the page, and with each little change, such as making a heading bold or adding some padding below an element, the page looked better than it looked previously. Within a day or two, the page was something that I’d proudly associate with myself.

So to close this one, let’s review the lessons we learnt:

  • Your product’s first iteration is going to look like shit unless you decide to launch it too late. Don’t get saddened by its lack of ‘perfection’, for you’ll have plenty of time to do it down the line. Focus on the basics like why you started it all and what purpose it is supposed to serve.
  • Separation of concerns is important. It is easy to get overwhelmed by the things to do, so have a plan and don’t put too many things on your plate at once.
  • It takes time to create anything worthwhile. You cannot get something right in the first iteration itself. It is against the laws of nature. Evolution required some four billion years to create the beautiful individual that you are. Don’t rush it beyond what is possible and practical.
  • In UI development, things are never done to a 100%. You try to improve as much as possible through iterations, meeting business requirements and personal standards and then going well beyond that. At a point, you call it a day and move on to the next challenge.

I hope you enjoyed this little article. Thank you for reading.

17 Tips After A Year In Javascript Development

Uncategorized

Long time no see, Javascript. Where had you been, my old friend?

Well, not really. I have been writing Javascript more than ever since my last post about the language on this blog, some 26 months ago. As some of you already know, writing Javascript has become my day job and it is honestly one of the better things that could’ve happened to me. I’ve learned a lot of the nitty-gritties of the language and surrounding environments (and the web) and gained invaluable insights about software development in general.

In this post, I aim to cover some of things I wish I knew as a Javascript newbie (no offense to anyone. I’m a newbie myself in most ways). These aren’t supposed to be comprehensive or even correct, for that matter. No particular order is followed. These are just things I feel I could’ve helped myself with while learning the language and building some initial projects and wish to share with you guys here.

It will be most helpful if you’re just finishing learning the core language and want to find out what exists in the world of web development. Feel free to contradict and correct me.

1. All Theory And No Practice

Get over with that college student attitude of reading chapter upon chapter of dull text and not writing a single line of code. And by code, I don’t mean the example code snippets. Go out there on Github and Reddit and search for beginner problems statements. Creating a simple TODO list or displaying the top posts on Reddit frontpage via their API doesn’t require any deep knowledge.

The essential skill to learn here is to be able to use a search engine, find relevant documentation, getting passive help from online communities (See stackoverflow or Reddit; r/javascript, r/webdev, r/frontend are good places to start looking). You cannot learn cycling by just reading about cycling. You learn cycling by riding a bicycle, by talking to your friends about the dangerous stunt you pulled off, by going offroad on the weekends, by watching the pros ride and then trying those things. That’s what makes it fun. Don’t make it a job. More often than not, you’ll suck at the end of it.

2. Build Tools

I made the mistake of assuming I knew everything about the language once I finished my guidebook. The reality was far from it. Language is just one little part of the puzzle of mastering any programming stack. There are many support tools that I didn’t care to explore back then. For example, there’s something called as a build tool.

In an organizational setting, you don’t start a project by creating an index.html and then linking it with your main.js. Instead, you make use of build tools (See Webpack or Grunt) to compile your code into a form understood by new and old browsers alike, bundle all of your code together and make it production ready. A basic understanding of build tools will go a long way and come in handy when making your next hobby project shine.

3. Modularization

Modularization, in simple English, means separating parts of your code that work similarly. Just like we sleep in our bedroom, bath in our bathroom and cook food in the kitchen, code the does similar things should be separated into different files and functions/classes. fetch() API calls in one file, little utility functions/hacks in another, DOM manipulation functions in another and event listeners separately. This is one of the many ways of doing it. See what works best for you. Remember that your frustration level two weeks later while finding a bug in your code is inversely proportional to the number of files and functions you separated your source code in.

The importance of modularization strikes only when you feel the pain in your back scrolling up and down and sideways finding a particular section of a function responsible for something. You’ll have to take a leap of faith here to learn this lesson the easy way.

4. Consistency

You don’t have to be collaborating with other humans to follow this. Just pick up a code style and stick to it religiously (see Airbnb’s Javascript style guideline for a starting point). You’d hate yourself if you open your code editor and find things like varNames and var_names mixed all over the place. This is not limited to just syntax or micro stuff. On a broader level, follow the same style for everything. If you return a Promise from an API service function, then stick to that style. Don’t interchangeably return promises and data, for example. And don’t mix imperative, object-oriented and functional coding styles randomly. Make conscious calls on what is right for the situation, and document appropriately.

The less you think about the language semantics, the more you can think about the problem you’re trying to solve. And consistency will definitely reduce a lot of the unnecessary load of deciphering your own code.

5. Documentation

Another task deserted of any love in the realms of software engineering. No one likes to document code, take it from me. But the difference between a rookie and a master is knowing when and why something must be done irrespective of his affection towards the job. Write comments, doc strings. Decorate your projects with nice READMEs and fancy “build passing” and “codecov” badges. No, you’re not doing it for others. You’re doing for yourself because you know it is right. You want to know your thought process two months later when you read the code to make some changes. It is like an “enable time machine access to this point in time?” button in your code, which you reply with a “deny” if you decide to not document your code.

6. Think, Think, Think And Then Write Code

This is one of my personal favorites. I’ll start coding 0.006 seconds (number might be slightly inaccurate) after hearing the problem statement. Don’t do this mistake. Don’t fall for your heart saying it has understood it all. Your heart is wrong. Think about the problem. Confirm that you have understood the requirements correctly. Formalize the requirements in your mind, or better yet, in a Github issue or something. Come up with a naive approach to do it. See if you can optimize the code, think about space-time tradeoffs, think if you can use something like a hashmap etc. You’ll be surprised how many better solutions you’ll come up with after thinking about it for 10 minutes as opposed to jumping right into the code.

7. Tracking Your Project’s Progress & Version Control

Use git, Github (or any other Git hosting). Use issues to write down what you plan to do for the next iteration. Use PR (pull request) feature to fix that issue. Write down if there were any caveats in the comments below the PR requests. Yes, do it even if you’re the only person working on the project. It might seem unnecessary and tedious, but once you get the hang of it, your time will be much better utilized and progress much better tracked.

8. Automate Your Deployment

I remember my workflow before starting work. Keep committing to the master branch and testing things out locally. And when things look decent enough, spin up a digital ocean droplet, clone the repository and start the development server there. That was my ‘deployment’. If any changes need to be done, commit to master and push from local. Pull them on the server and restart development server.

Now that I know better, I not only make separate branches for each feature, I set up auto deployment on Heroku on particular branches (mostly master and staging/development). It’s a one-click affair (assuming you followed the build tools part) and takes away a lot of hassles (and is free as well). Knowing the right tools can make a huge difference, as you just saw.

9. Unit Tests

If you’ve ever worked on a non-trivial project (say around a thousand lines of code), you will recognize the tragedy when you modify your code while adding a new feature and accidentally break a lot of the old code which you realize later when things start going south at which point a lot of git-fu is required to get back to a safe point and not lose any newer work. Let me tell you something. It is all avoidable with simple unit and integration tests. Think of tests as employing a bunch of robots to look over your old code and point out when you’re about to break your own (or someone else’s) code.

It is very hard to convince yourself to write tests even after knowing the pluses, let alone without knowing them. But again, you’ll have to take a leap of faith here. Or you can wait until someday while casually refactoring some code you mess something up which you realize only after three days with hours of debugging which could’ve been avoided by writing a simple test.

10. Use ES6

This might be a no-brainer to most of you, but I still find people writing older Javascript syntax even though they know and understand the newer spec. With modern build tools and polyfills, most of the times not writing the latest spec syntax is just laziness. If you find yourself writing things the old way, force yourself to use the newer ways. They are there for a reason.

Let’s be honest guys, Javascript is not the most elegant language. The older versions are riddled with bugs in the spec itself (see function scoping of var, the this keyword, double equals (==) comparison and automatic type coercion). The newer versions of the language are an attempt at fixing things and we should respect that, even if that means going out of your way to follow the newer semantics.

12. Understand What JS Is And What It Isn’t

Javascript, the core language, is actually a subset of the things we’re used to of using in our everyday development on the frontend. The core language is relatively small and simple, but add DOM APIs to it, and suddenly it becomes something slightly bigger and a bit more complex. Javascript is single threaded but DOM APIs like fetch (or xhr) and setTimeout are asynchronous because of the browser’s magic. Learn to consciously differentiate between them as that will come in handy. As a general rule, DOM is very slow as compared to the Javascript language engine.

13. You Probably Don’t Need That Framework

This has been repeated enough that I could’ve skipped it, but I’ll still put it out here. If you find yourself importing Lodash or Underscore for simple things like mapping, iterating over objects, filtering objects and things like that, consider looking for equivalent core Javascript alternatives. The latest ECMA spec adds a lot of helpful data structures and utility functions that make writing functional code very easy and pleasant.

Similarly, if you are planning to use ReactJS or Vue just because everyone seems to be talking about it, don’t. Go read up their introduction and documentation and judge if your project fits the use case that particular framework is build for and the problem it is trying to solve. For learning purposes, you can consider using the frameworks for trivial apps, but for anything else, make decisions on a strictly use case basis.

14. Learn Basics Of CSS

While the popular opinion still says that CSS is a pain in the back to master, I’d suggest learning almost 60-80% of it with maybe 20% of the efforts. No need to remember which browser supports what niche attribute. Just the important very-handy-in-real-life parts like flexbox, grids, box model, margins, paddings, positioning related, display related, floating related and so on. You will cover a lot with just these, and they come in super handy for personal projects. (Again) trust me.

15. Explore Web Workers/Service Workers

What if I tell you, you can hijack API requests from your webapp and service them without making a roundtrip to the server? You’ll say, if the request is serviceable on the frontend itself, why bother making an API call in the first place, and I make a poker face to that. Jokes aside, service workers enable you to handle network connectivity drops and provide a much smoother experience. You can save the data that a client wanted to send the server in the browser while offline and then replay those requests again when network connectivity is recovered.

Web workers allow your code to delegate work to another processing thread. This can be any type of work that you would want to unload off the main thread like heavy calculations etc.

16. Explore Browser-Side Databases

There’s localstorage, there’s cache, there’s indexDB. This is apart from the cookies and session headers. Basically, there is no shortage of storage options on the client side. Try and use these to cache and store data that need not be sent to the server, or maybe cache in case of uncertain network availability. With service workers and Cache, you can fine tune what gets cached and the behavior when a cache is hit or miss. You can even set up caching strategies with plain Javascript code logic.

There are wrappers such as PouchDB that can sync with the server side CouchDB and provide nice real-time syncing capabilities. Speaking of real-time, you might also want to check out Firebase by Google. I used to use it in the past and it was very easy to set up and work with.

17. Explore Progressive Web Apps

Now you can make your website work offline too. Ever seen that “Add to homescreen” button on certain websites? When you add those websites to your homescreen, you can then open them like regular Android apps and if they’re built that way, even use them offline (further reading: developers.google.com/web/progressive-web-apps).

In Closing

I hope this article was useful. If you’re a beginner with web engineering and have just finished learning the Javascript language, you might want to explore some of the above points in detail. I’m sure you’ll have fun building your next awesome project using these guidelines.

Thank you for reading.

OverTheWire Bandit 27-33 Write-up

Uncategorized,

The last part of the Bandit challenges was relatively easy with most of the flags attainable with basic git knowledge, except for the last restricted shell escape. Try them here: OverTheWire Bandit

Bandit 27-28

This is as simple as it can get at this stage. Just clone the repo and cat the README.md file. The flag is in plaintext.

Bandit 28-29

In this stage, if you cat the README.md file, you’ll find xxxxxxx in the place of the flag. If you do a git log, you’ll see that the password was entered and then removed. Just checkout the previous commit with git checkout {hash} and you’ll have your flag in the README.md

Bandit 29-30

There’s no commit history this time, and the README.md file says “no password in production”, which is a clue. Do a git branch -r and you’ll see a development branch. Checkout into it (git checkout dev). cat README.md in this branch to get the flag.

Bandit 30-31

No password in previous commits or branches here. But if you do a git tag, you’ll see a tag called “secret”. Do a git show secret and you have your flag.

Bandit 31-32

Add and commit any random file, remove the wildcard entry from .gitignore and push origin. The flag is in the verbose output of the commit.

Bandit 32-33

This is a restricted terminal escape challenge, very interesting. I urge you to think of creative ways of loopholing this before looking at the solution.

So the terminal converts every command into uppercase before executing. So ls becomes LS and cd becomes CD and nothing works.

One way of loopholing this behavior was symlinking a helper binary to an all caps name. I choose vim for the purpose, but cat, less or more, anything would’ve worked. Symlink the binary in your temp directory in some all caps name.

$ ln -s /usr/bin/vim /tmp/mytempdir/VIM

Now, simply running ./vim will execute VIM and you can then read the flag file with :r /etc/bandit_pass/bandit33 in vim.

Thank you for reading

HTTPS By Default

Uncategorized,

There was a time, not too long ago, that I used to dream of running a website that had HTTPS written in the address bar. Most big websites had it, but very few little ones and personal blogs did. I couldn’t because it required a lot of money (for a student, at least). Today, thanks to the efforts by Let’s Encrypt, Cloudflare and many other organizations which gave away basic SSL certificates for free, all of the domains I own are on HTTPS. But that’s not even the best part.

The best part is that HTTPS is now seen less as a luxury by small website owners and more of a necessity. Part of the reason for this is Google’s penalty in terms of search rankings for websites without HTTPS. The other part, and this is more important for the little non-business blog owners who do not care much about traffic and SEO, Google Chrome (>= 68) has started displaying a Not Secure for non-HTTPS websites. I would say that Google is doing a great job at providing people an incentive to switch. I hope Mozilla Firefox follows in Chrome’s footsteps in this regard.




non-https website on Google Chrome


non-https website on Mozilla Firefox

On similar lines, Github now only shows the entire web address (along with the protocol) if it is on HTTP. If the link posted is HTTPS, Github will truncate the protocol part and only show the domain name.

For example,

https://example.com

will be rendered as

https://example.com

while

https://example.com

(notice https) will be rendered as

example.com

If you notice, this is opposite of what happens in browsers today, http is truncated and https is emphasized.

I personally hope more and more website and browsers treat HTTPS as the default and HTTP as the exception, and not the other way round. The merits of using HTTP over HTTPS are either obsolete or negligible (read more about HTTPS here: ELI5 – How HTTPS Works). On the other hand, if done properly, HTTPS can be much faster than HTTP.

We already know that the more people using encryption, greater will be the overall value of using it. I’ll end here with a quote by Bruce Schneier, a cryptology badas- expert.

Encryption should be enabled for everything by default, not a feature you turn on only if you’re doing something you consider worth protecting.

This is important. If we only use encryption when we’re working with important data, then encryption signals that data’s importance. If only dissidents use encryption in a country, that country’s authorities have an easy way of identifying them. But if everyone uses it all of the time, encryption ceases to be a signal. No one can distinguish simple chatting from deeply private conversation. The government can’t tell the dissidents from the rest of the population. Every time you use encryption, you’re protecting someone who needs to use it to stay alive.

https://www.schneier.com/blog/archives/2015/06/why_we_encrypt.html

Thank you for reading.

System Stability & Moving Back To XFCE

Uncategorized

One thing that I really hate, and I don’t use that word very often while describing my computer preferences, is system crashes. It’s one of those things; just unacceptable to me. You’re working on something important, and all of a sudden, the DE (Desktop Environment) decides that it needs to restart itself, and you lose all of your windows, terminals and most importantly, context. Coming back from there is a 15-minute process in itself; logging back again, starting the browser, IDE, terminals, entering virtual environments, running test servers and so on. As you can tell, it can escalate from slight inconvenience to very frustrating in little time.

When I got my new laptop back in May, I decided to switch away from XFCE. To be honest, I did try installing XFCE but couldn’t due to some issue starting the DE. Since this was a fancier laptop with better hardware, I assumed I can afford running a somewhat heavier DE for a better user experience (and my colleagues’ Macbooks constantly reminded me that I’m using an ancient (looking) DE). I did some research and was split between KDE and GNOME 3.

So the initial impressions of GNOME 3 were not very convincing (not that this was the first time I tried GNOME 3 anyway). I never liked the gesture-like way of accessing windows and quick menu. I’m more of a click-click person. But I decided to stick with it and see how it goes, customizing whatever that I can. So after that, things went uphill for a while. The more I used GNOME, the more I started to appreciate it. I brought back the ‘conventional’ application menu, quick access bar on the left side with an extension called Dash to Dock, Pomodoro and a bunch of widgets for the top bar (which by default is mostly empty).

A few issues persisted from the beginning. The most important one was memory and CPU usage. I looked up and concluded that it is a general problem and not just my laptop. The problem is not just the high usage of system resources (which is even a good thing if you trust the kernel). Problem is when you see gnome-shell constantly use one CPU even while idling and 500MB-1GB of data just after startup. Now, due to this, I was constantly facing situations when the RAM would go over 90% and the system would start to lag. This was serious, but this wasn’t the worst part.

I could’ve lived with a (little) laggy system, a system that lags while opening the app drawer, for example (tip: create shortcuts to all the apps that you frequently use to avoid GNOME 3’s app drawer altogether), but the DE would also crash all of a sudden, wasting my time re-spawning everything. And it was especially bad when it happened during my work hours. That was a deal breaker. I tried to debug it, but couldn’t convince myself to spend more time on it as it wasn’t making a lot of sense. I installed XFCE, made it work and it felt like I’m back home to my countryside house from a vacation in the city. Felt good.

In conclusion, I think I’m biased here. I had a preconceived notion about GNOME 3, and I might have fallen for that. Maybe GNOME 3 is objectively better at many things that my bias didn’t let me see. Don’t get me wrong. GNOME 3 is a wonderful DE, and for someone who values the bells and whistles that come with GNOME (I had three finger and four finger gesture support for once in my life. Thanks, GNOME) I think it is a perfect choice. For me, however, system stability is way too important than any secondary convenience feature.

An interesting thing I saw Mac OS users do was that they used to always suspend their machines, not shutting them down often. I wanted to do that since forever, just close the lid and be done with it. I never could do that on my old laptop because new issues would creep in after resuming from suspended state (failure to connect to wifi, display staying all black, USB ports not working etc to name a few). No such issue is present on my Thinkpad, and as a result, I suspend it in between use and at night. The system is rock solid, even at heavy loads. As an enthusiast, it gives me a lot of pride in mentioning my last shutdown was nearly ten days ago.

22:00:09 up 9 days, 11:17, 1 user, load average: 0.84, 0.58, 0.59

I’m sure a lot of you reading this can relate to the pride of showing off uptimes and talking about system stability, or the joy of keeping your car running in top notch condition after years with proper service and care! This is similar. Hope you found this interesting. Thank you for reading.

Backend Development With Flask

Uncategorized,

Context

I distinctly remember how much I liked writing backends (well, I tried). I also remember thinking, I will never become a frontend engineer. Ever. Like many software developers in my circle, I hated writing CSS, and JS hadn’t clicked until then. And these were the days before I knew anything about automated deployments. Deployment, for me, was spinning up a Digital Ocean droplet, installing everything manually, setting up the database (and copy pasting the db creds into code), and then running the development server and keeping it running. Stop laughing, please.

Motivations

So I decided to pick up some backend skills again. The main reasons for this were that I’ll need a full-time job soon, and it would be much better if I get to write the full stack of the web instead of just the frontend. Secondly, knowing full stack is a superpower that I’d not want to not know. It comes in incredibly handy when working on some personal projects that require the web as an enabler. Thirdly, I wanted to refresh my Python skills. I liked python a lot back in the day, but I had lost touch since the last couple of years. So with those goals, I started with Python and Flask.

Python & Flask

I liked flask, but I’m still struggling with some basics, especially working with app instances for writing tests. Yes, there are a few differences working with the backend this time versus like three years ago. I’m following (or trying to follow) the best practices, writing tests for the code, I have a nice CI pipeline which starts with testing of the code and ends with deploying the app on Heroku. Most importantly, I have an excellent mentor for my backend adventures who’s a badass python programmer.

Flask is fun to work with. The framework is minimal, kinda like React. There’s a lot of support online, and there are great plugins already available for most common functionality. Databases are one of my weakest points in web engineering, and I’m trying to experiment a lot of things on the model layer with SQLAlchemy and Postgres backend. One more novelty for me was asynchronous programming. In Javascript, you had to beg for things to be synchronous. But here you face a different problem; If something is slow, then the entire thread is blocked. For taking care of things that are slow, say sending an email, one could use Celery with RabbitMQ’s backend. All of this is given to us ready-made by Heroku. So no more manual DevOps work, and fewer variables to worry about.

The other motive was to learn quality Python 3. In python, you have a pythonic and many non-pythonic ways of doing things. There’s no point in writing Python like C. I wish to learn the philosophy of the language so that I can make the right choices when deciding how to solve a problem. There’s nothing like writing clean and elegant code that others can appreciate. In the last week, I also got exposed to a lot of different data structures that the python library provides for specific use cases. In the right scenario, using an appropriate data structure can be the best optimization you can do to your code. I am looking forward to getting a hold of this as well.

Lastly, one thing that I never thought I’d so, but I’m doing, is trying to learn object-oriented programming. I had done some OO python in the past, but it never clicked. Laster, with JS, it was all about functional programming. Now, I’ve rediscovered OO programming and wish to relearn it, apply it and try to make it click. I like the contrasts in both paradigms of programming, and it could not be better explained than this StackOverflow answer.

  • Object-oriented languages are good when you have a fixed set of operations on things, and as your code evolves, you primarily add new things. This can be accomplished by adding new classes which implement existing methods, and the existing classes are left alone.
  • Functional languages are good when you have a fixed set of things, and as your code evolves, you primarily add new operations to existing things. This can be accomplished by adding new functions which compute with existing data types, and the existing functions are left alone.

Overall, I feel I’m becoming a little (very little indeed) mature with programming. Instead of sticking to paradigms and trying to defend the one that I’m most comfortable, I’m trying to see why those paradigms exist and what problems are they helping me solve. And since python supports both object-oriented as well as functional programming, it will be fun to work on any such problems.

I would write some technical articles on the subject when I feel confident enough in the near future. Just wanted to give you an update on what’s happening on my front in this one. Hope you found it useful. I’ll leave you with an interesting video about ‘Duck typing and asking forgiveness, not permission’ which is a design pattern in Python. Thank you for reading.