Instance#2: Twitter Layoffs, Kafka In-Sync Replicas, The Language of Pods and More
Also, actionable Deep Work Strategies for Software Developers
Creating new instance…
Fetching dependencies…
Instance created successfully!
Hello, this is the ProgressiveCoder Publication and welcome to a new INSTANCE of our weekly newsletter.
Today’s edition is packed with lots of relevant content. Some of it a little sombre.
Massive Layoffs at Twitter
The mysterious concept of In-Sync Replicas in Kafka
How do you describe a Kubernetes Pod?
How can software developers do Deep Work? (3 actionable strategies)
And a little bit more…
Twitter Layoffs
Over the weekend, the industry has been hit by news of massive layoffs at Twitter.
On 26th October, Elon Musk finally took control of Twitter after a lengthy and controversial acquisition saga. Immediately, rumours began circulating that Musk plans to cut Twitter’s workforce by as much as 50% to 75%, though Musk publicly denied any such plans.
Musk’s statement has turned out to be false. Over the weekend, company-wide layoffs have played out at Twitter.
Of course, layoffs are nothing new in the tech industry. However, the approach to layoffs at Twitter has taken everyone by surprise.
On Friday, all employees received emails instructing them to return to their homes if they are on their way to office or already in office. They were told that by the end of the day, they would receive emails (company email id if they survive or personal email id if they are out) about their future.
Below are the type of emails we are talking about. These ones are most probably fake but you might get the idea.
For an organization like Twitter, all of this sounds quite unbelievable. As the dust settles, it appears that nearly half of the employees have been let go.
Layoffs are incredibly hard. I have had the displeasure of witnessing a layoff unfolding from quite close.
Though I was on the safe side, the feeling is quite uncomfortable. Colleagues you had known for years are gone without even a final goodbye. In most places, the ones fired aren’t even allowed to return to their desks and are escorted to the exits.
Even though your mind tries to find a rationale for why a particular person was fired, you aren’t able to figure out a valid reason. Usually, such layoffs are more because of business reasons and have very little to do with the employee’s actual performance.
As you feel for what your colleague might be going through, you can’t help but experience a sense of fear in case you ended up in the same situation. What if there are more such plans in progress? Where do you stand?
Things are made even worse by the management trying to package up the whole affair into some sort of a grand strategic pivot.
I saw the below tweet that seems to sum up this feeling.
Of course, Elon Musk has justified his decisions saying that Twitter had become bloated and is losing a lot of money. There was no way for him to turn it profitable without making these hard decisions. It is hard to validate the sincerity of these claims.
Kafka In-Sync Replicas are Important!
You already know Kafka partitions are replicated. If not, just check out the previous edition for a quick recap. You will get an idea about leaders and followers and all that sort of stuff. It’s not a big deal.
The big deal is the existence of Kafka In-Sync Replicas.
Kafka In-Sync Replicas are the replicas of a partition that are currently in-sync with the leader.
In other words, a replica is considered in-sync if it has fully caught up with the leader partition in a certain amount of time. The time setting is controlled by a property replica.lag.time.max.ms and can be set at a topic-level.
This sounds all fine. But what are the implications?
Firstly, a leader is always an in-sync replica. This is quite natural since a leader would always be fully caught up with itself.
However, a follower is an in-sync replica only if it has fully caught up to the partition it’s following. In other words, it can’t be behind on the latest records for a given partition.
So what happens if a follower fails for some reason?
Followers replicate data from the leader to themselves by sending periodic fetch requests. If a follower fails, it will stop sending these requests and eventually fall behind. After a set amount of time, it will be removed from the list of in-sync replicas.
In the above example, Broker 1 and Broker 2 are in-sync replicas of partition 1.
However, due to some reason, Broker 3 has not been able to replicate the partition’s data and hence, it is kicked out of the exclusive list.
Is that a problem?
Yes, a massive problem. You see — it is perfectly possible that despite being configured for replication, a partition can end up with only one in-sync replica i.e. the leader.
Essentially, the partition can become un-replicated. This makes our setup vulnerable to data loss — something which defeats the very purpose of replication.
So - how do we make sure there is no data loss?
We will cover it in the next edition. But do answer this little quiz. It won’t take long.
How do you describe a Kubernetes Pod?
You know all about pods in Kubernetes. They are quite popular these days. In case this is the first time you have heard about them, consider this comparison.
In the animal kingdom, pods are meant to signify a group of whales. In the world of software, pods represent a collection of application containers. As simple as that!
So, how do you actually describe a Kubernetes Pod? Not for humans but for the Kubernetes cluster.
You create what’s known as the Pod Manifest. Sounds fancy but it is essentially a text file representation of the underlying object.
Here’s a very simple example of a manifest file:
apiVersion: v1
kind: Pod
metadata:
name: basic-pod-demo
spec:
containers:
- image: progressivecoder/nodejs-demo
name: hello-serviceYou can write this file in YAML or JSON format. The above example is in YAML.
YAML is generally more preferred because it is slightly more human-editable and also supports comments.
Each line within the manifest holds some significance:
The
apiVersionspecifies the Kubernetes API version we want to useThe
kinddenotes the type of resource (in our case, Pod) described by this file. There are many other resources we can create and the value ofkindchanges accordinglyThe
metadatasection contains thenameproperty. This is the name of the pod.Lastly, the
specsection is meant to provide details about the containers within the pod. Containers need a name and the image they are supposed to run.
And that’s about it. We can create a pod using this manifest file.
$ kubectl apply -f basic-pod-demo.yamlIf everything went fine, you’d see details of the running pod when you execute kubectl get pods.
NAME READY STATUS RESTARTS AGE
basic-pod-demo 1/1 Running 4 (43s ago) 88sOf course, this is just the first step. There’s a lot to unpack with Pods and we’ve barely scratched the surface.
If you are interested to know more, please refer to this detailed post on Kubernetes Pods.
How can Software Developers do Deep Work?
In the previous edition, we talked about the concept of Deep Work.
To recap, Deep Work means professional activities performed in a state of distraction-free concentration that can push your cognitive capabilities to the limit.
The question was - how can we do Deep Work? Are there some strategies we can follow?
Deep Work requires a significant commitment to pull off. As a result, its strategies should also reflect the same.
Here are a few strategies that can help you pull it off:
1 - Build Rituals
The benefits of Deep Work increase the longer you can continue in the depth-mode.
Rituals can help minimize the friction while transitioning into this state of depth.
Try to determine things like:
Where you’ll work and for how long. If you have a cabin, try to put a “do not disturb” sign during your Deep Work sessions. For software developers, this might mean logging out of Skype or any other instant messaging software. In open-office setups, see if you can get a conference room for yourself to carry out activities that require Deep Work.
How to work once you start? Think about rules such as not accessing the internet. Of course, this might not be feasible as all developers need StackOverflow! At least, try to stay away from social media sites such as Facebook or Twitter.
Supporting the Work. Maybe develop a ritual of starting your work with a cup of coffee or keeping some snack handy. Ultimately, you brain will start correlating the act of drinking coffee with Deep Work.
2 - Don’t Work Alone
While this might sound counter-intuitive, you can also support Deep Work by working in pairs or even a small group. Group work creates an environment of back-and-forth and leverages something known as the whiteboard-effect.
Some types of problems (especially the hard ones) lend themselves quite well to this style of working. Working with someone can help you delve deeper into the task than working alone. The presence of another person waiting on your next input forces you to focus.
For software engineers, this strategy aligns itself pretty well to the common practice of pair programming. Also, architects often use this strategy of group-thinking for finding solutions to complicated design problems.
3 - Execute like a Business
A business should be efficient in order to survive. If we execute things like a business, we can also increase the chances of success for our Deep Work endeavour.
Some common things we should keep in mind:
During Deep Work, always focus on the wildly important goal. Try to make the goal itself ambitious.
Act on the Lead Measures. Basically, don’t focus only on the outcome but also the input. For example, if your wildly important goal is to finish X number of story points during a sprint, your focus should be on how many hours you actually devote to Deep Work.
Keep a Scorecard. Make a physical artifact storing data about your Deep Work efforts. Keep marking the number of hours you spend doing Deep Work every day. Your mind tends to play well when some sort of scoring is involved.
Develop Accountability. Perform a weekly review of your Deep Work efforts. Celebrate good weeks. Introspect the reasons for why certain weeks weren’t good enough. Make appropriate adjustments for upcoming weeks. Rinse and Repeat.
Though there is a lot more to Deep Work as a philosophy, I believe these strategies and steps can help you get started.
Maybe, you are already doing a bunch of these things and just need to fine tune stuff.
Five Movies that Programmers should watch
To cap off today’s edition, let’s have a little bit of fun and end on a good note.
Though there are thousands of movies that one might enjoy, here’s a list of five movies I think every programmer should watch once.
1 - The Matrix
Let’s be honest here! Matrix makes software engineering look cool despite the horrible implications of advanced AI that it tries to depict.
If machines can enter our brain and make us really feel like a kung-fu master or a superman, AI domination doesn’t sound too bad!
2 - Jurassic Park
Hear me out on this one first. I know that programmers may not really relate to all the bio-technology jargon introduced in this movie
But I’m not talking about the bio-tech experts. I’m talking about our sole representative in the movie. Mr Dennis Nedry.
Despite his questionable intentions, the guy was a genius. With 90s tech, he built all the elaborate systems of the park and just like every other non-techie boss, Mr John Hammond didn’t appreciate his talent and PAID HIM LESS!
On this one, I’m definitely on Mr Nedry’s side. And I hate that dinosaur who killed him!
3 - Minority Report
A movie so cool that you had to watch it multiple times to fully understand what’s going on.
Minority Report really shows the remarkable things technology can achieve and how it can still probably be a bad idea.
It also shows you the perils of relying on a prototype program in production. Haven’t we all done it sometime?
4 - I, Robot
This was the movie that showed the true power of robotics technology and gave us a glimpse of machine learning before it had become common place.
But it also taught one important lesson.
At times, it can be extremely difficult to code simple if-else conditions!
5 - The Social Network
This is one of my favourite movies about the software industry.
The great part about this movie is that it leaves you with different impressions as you grow up.
When I first watched it, I was hooked onto Facebook and the general idea of a genius developer who created a fascinating social network out of thin-air. You think others are trying to put Zuckerberg down because they are zealous of his achievements.
As you grow older, you realize that there is no such thing as a genius developer. Or, at least, not in the way depicted in this movie.
The idea may be brilliant but executing it on a scale as large as Facebook requires the contribution of thousands of developers.
And maybe, the others weren’t completely wrong when they accused Zuckerberg of playing politics.
That was all for today’s edition.
If you liked it, consider sharing it with your friends and colleagues. In case you have any suggestions and ideas, please don’t hesitate to write them in the comments section of this post.
See you until the next time!