Manage a large white-list with Shorewall and IPSets

I no longer screw my way around with IPTables when I want to run a firewall. My go-to firewall tool on Linux is Shorewall. It’s great, powerful, and simple to use.

Last week somebody at work brought my attention to a Linux server I had had set up. The problem was that the commands “shorewall check” and “shorewall restart” were taking approximately three minutes combined to run. I thought that was needlessly long. I suspected resource exhaustion issues. There were none. The disk was good, inodes plently, CPU hardly being used, the network idling, and plenty of RAM was available. What was slowing Shorewall down?

On that server, the process of adding an IP address in the white-list was automated by an API and the use of Shorewall macros. The API call would slip the IP address into the macro file, and then ask Shorewall to read the changes. Over time the macro file had grown to a whopping 15MB size. That was huge for a text file that contained nothing but an IP address and accompanying comment for each line, with almost 200k+ lines in it. Because the use-case was genuine, for us at least, I didn’t see why Shorewall should take that long to process a big macro file.

Continue reading

Advice to CEOs

Jacques Mattheij, while giving quality advice to potential and existing CEOs last week:

When the company is doing ok, there is money in the bank, when you’re hiring and the sun is shining anybody can be CEO of a company. But when the bank accounts are running dry, when you have to fire people and when it seems as if there is just no end to the bad news it really matters who is in the driving seat. It seems such a great thing, to be the master of a company but it is a double edged sword. With all that freedom and executive power comes the flip side of that coin: executive responsibility.

Gravitational waves!

I read about the news of a possible detection of gravitational waves a month or so ago. When I dug further, I also came to understand what “blind injections” were. In large observatories with equipments that can detect extremely sensitive distortions in space-time such as LIGO, people quietly inject distortions into the system to ensure that the systems and people monitoring them can really actually detect anything if something were to pass through. As I read further, I found out that there was a good chance that what was detected may have been only a blind injection. 

The equipments they have at the LIGO observatories are called the Interferometers and are a work of art. What they do with Interferometers is that they take a laser beam and split it in two parts, and send both parts in two tunnels at ninety degrees (so, essentially L shaped). At the end of the tunnels, mirrors bounce the beams back. When they come back, both beams interfere. They can interfere constructively, in which case the resulting light is brighter, or destructively, in which case there is no light at all.

The Interferometers are designed such that the two beams always perfectly interfere destructively, resulting in no light.

But this time, they detected a faint light, which was the gravitational waves from two very powerful black holes colliding and merging into one, distorting space-time to cause the two laser beams to interferer constructively.

To think that Einstein theoretically described these gravitational waves a hundred years before they could be faintly detected is such a big thing.

It is an important confirmation for both cosmology and astronomy. Up to this point, cosmologists tried to observe the universe via electromagnetic waves, which take the shape and form of light. But this severely limited their reach, causing them to not be able to observe/detect blackholes, because blackholes don’t allow electromagnetic waves to be emitted into the far universe. They do, however, release gravitational waves all the time, as do every other object in the universe moving about.

With the ability to detect gravitational waves, a new era in astronomy and cosmology begins today, where we can now start to learn more about and discover black holes and similar objects.

This thread in /r/askscience tracked the entire press release yesterday and provided information about the whole subject.

A dog chasing a cat up a tree

With the economy tanking and the morale of the general population spiralling downwards, it is not uncommon to find a shocking abundance of lethargy about the wild animals that inhabit this country of ours. When was the last time you saw a cat in your house or in your neighbourhood chase down a mouse? Beyond depictions in cartoons, have you ever seen a dog chase a cat up a tree and wait with a sullen look underneath it for the cat to come down?

Today, after getting off from the parking lot, I passed a listless wild dog sitting against a whitewashed wall. I made the kind of sounds I make to attract a lovable cat we have at home. It wagged its tail but ignored me. Just then, on top of the wall against which the dog was resting a cat made a run for its life, being chased down by a stick behind the wall. It ran over broken pieces of glass mounted on top of the wall, until it made a jump to land not far from where the dog lay. No sooner had the cat landed on all of its legs than it made a frantic dash across the road, followed closely by the same dog that was lazily lying about. They both sprinted across the road, and within what must have been a few seconds the dog managed to chase the cat up a thick tree across the road. Because I was walking past the same tree and watching them both animatedly, I came to a halt in order to take a look both at the cat who cemented itself on top of the first branch, claws fiercely gripping it, and the dog who stood still under the tree, unsure what to do. The dog made an attempt to walk towards me after a while, but I told it to fuck off. It continued to look up the tree, aimlessly.

SHA-1 goodbye!

You will be as surprised as I was when I found out to know that starting next month, most of encrypted web–web that is behind SSL–will stop working for a majority of users with browsers and devices that don’t support SHA-2. SHA-1, the no longer de-facto hashing algorithm, will stop working in ten days or so.  As CloudFlare commented on their excellent post on the subject, this represents a big problem:

Unfortunately, this list largely overlaps with lists of the poorest, most repressive, and most war torn countries in the world. In other words, after December 31st most of the encrypted web will be cut off from the most vulnerable populations of Internet users who need encryption the most. And, unfortunately, if we’re going to bring the next 2 billion Internet users online, a lot of them are going to be doing so on secondhand Android phones, so this problem isn’t going away any time soon.

Killing SHA-1 is a good move. They did it first with MD5, SHA-1’s predecessor, but it took everybody a couple of years to finally get rid of MD5 from the face of the web. It wasn’t an insurmountable task then, because browsers and devices at that time also supported SHA-1. We are not lucky this time, though. If SHA-1 is killed starting next year, a lot of people will be left without access to a lot of websites over SSL.

Mozilla did it, and paid a heavy price.

Yes, please don’t change SSL certs on without checking with #www or #webprod as we killed 1 million downloads recently by switching to SHA-2. A lot of the world is still running old browsers and come to our website to get Firefox.

It is a serious enough problem that CloudFlare, Facebook, Qualys, and Mozilla have addressed it, and workarounds for it, in detail.

I can only hope that more companies come out to talk about and address this issue.

Which version of Kibana is compatible with which version of Elasticsearch

I have been playing with Elasticsearch and Kibana at work. In my particular use-case, there’s data coming into Elasticsearch through Sensu. A customised third-party Graphite handler is used to feed data into Elasticsearch via its API. Because Elasticsearch v2 was recently released, I upgraded Elasticsearch and left Kibana running at v4.1.2. When data feed was established, I could verify corresponding document objects being created inside Elasticsearch. What I couldn’t figure out though was why the ‘Discover’ tab in Kibana did not show any data despite having the correct index configured. What was even more perplexing was that I could correctly create visualisations based on the data I knew was in Elasticsearch.

One of the operators in the FreeNode IRC channel for #kibana pointed out that Kibana v4.1.x was not compatible with Elasticsearch and that I had to upgrade Kibana to v4.2. This is what I love the most about FreeNode and IRC.

With my problem fixed after upgrading Kibana, I set out to find information about which versions of Kibana were compatible with which versions of Elasticsearch. My search brought me to this support matrix on the official Elasticsearch website. In particular, the product compatibility section on the page provides a non-exhaustive list of versions of Elasticsearch that are compatible with versions of a number of different applications that work with Elasticsearch, including Kibana.

Hitler’s worldview and Lebensraum

In his book, “Black Earth: The Holocaust as History and Warning”, Timothy Synder has laid bare his diagnosis of Hitler’s worldview. This book has quickly found its way to the top of my to-read list after reading an interview Synder gave to an editorial fellow at The Atlantic, titled “Understanding Hitler’s Anti-Semitism“. Any explanation I attempt of that interview will fail to do justice, so therefore I choose to share some excerpts from it:

Hitler is often depicted as the prototypical totalitarian—a man who believed in the superiority of the German state, a German nationalist to the extreme. But according to Snyder, this depiction is deeply flawed. Rather, Hitler was a “racial anarchist”—a man for whom states were transitory, laws meaningless, ethics a facade. “There is in fact no way of thinking about the world, says Hitler, which allows us to see human beings as human beings. Any idea which allows us to see each other as human beings … come[s] from Jews,” Snyder told me in an interview. As Snyder sees it, Hitler believed the only way for the world to revert to its natural order—that of brutal racial competition—was to eradicate the Jews.

I have read assorted novels that touch The Holocaust and Hitler’s Nazi Germany to varying degrees, yet I had not thought of Hitler’s worldview in this way until now. To think that what Hitler instigated and then unleashed upon one half of the world was not simply a result of his nationalistic desires and pure hatred towards the Jews is mind-boggling. Far from it. For example:

So what Hitler does is he inverts; he reverses the whole way we think about ethics, and for that matter the whole way we think about science. What Hitler says is that abstract thought—whether it’s normative or whether it’s scientific—is inherently Jewish. There is in fact no way of thinking about the world, says Hitler, which allows us to see human beings as human beings. Any idea which allows us to see each other as human beings—whether it’s a social contract; whether it’s a legal contract; whether it’s working-class solidarity; whether it’s Christianity—all these ideas come from Jews. And so for people to be people, for people to return to their essence, for them to represent their race, as Hitler sees things, you have to strip away all those ideas. And the only way to strip away all those ideas is to eradicate the Jews. And if you eradicate the Jews, then the world snaps back into what Hitler sees as its primeval, correct state: Races struggles against each other, kill each other, starve each other to death, and try and take land.

He believed so furiously in the natural order the world must take that he risked his Germany and his German people to the brink of defeat before edging them off. How do so few of us concoct such beliefs and grand ideas, unthinkable to the lot of us? Where do they develop such thought process?

It is a riveting read. I am hoping that the book will prove no less.

On a very related note, I came across the Generalplan Ost. The ferocity with which Nazi Germans believed in the Lebensraum is mind-numbing. To my tiny brain, the very idea of Lebensraum is inspired heavily from the works of Darwin, particularly Hitler’s view of the world that portrayed it as ring in which races contest each other aggressively for the right of survival.

Regrets of the dying

Came across this post on Hacker News today. The link to the article was dead so found a cached copy.

There is a short piece from Paul Graham on the same topic.

What powerful thoughts. It boils down to the following things people who were dying regretted not doing in this lives:

  • Don’t ignore your dreams
  • Don’t work too much
  • Say what you think
  • Cultivate friendships
  • Be happy

As Paul Graham writes in his article, these should be on the top of your TODO list.

The unfortunate realisation I’m struck with is: I am missing on ALL of these things.

Ignoring my dreams. Working too much. Unable to say what I think from fear of reprisal, upsetting people, and/or hurting them. Missing out on friendships. Not being happy, blaming it on being inherently unhappy.

I will die one day with all of these regrets.

Tech Sabbath: Week #2

Last Sunday I set a challenge for myself. I also decided to write about the progress I will make every weekend. The following is a passing account of how I fared during my second Tech Sabbath.

This Sunday, in terms of the challenge, wasn’t very different from the last. For the most part I managed to stay away from my devices. Successfully. I didn’t turn off Internet on any of them, though, as I did last time, but refrained from using them. It also helped that nobody attempted to contact me via my devices. Let’s keep it that way.

I read several pages of Shutter Island. After putting it down last Sunday, I didn’t pick it up again until today. Because of being in the early stages, the book is moving along very slowly. I can only hope that it is just as riveting and mysterious as the movie.

A good part of my day was consumed by physical chores, and by the end of which, I was happy I got around to dealing with them. Procrastination has become the sad order of the day, and more so when it comes to household chores. It is easier to put them off with an excuse.

It was the Roland Garros final between Novak Djokovic and Stanislas Wawrinka. I cannot imagine how I completely forgot about it. Instead, I played Far Cry 4 on my XBox for a couple of hours. For someone who is obsessively involved with playing as well as watching tennis, I should think that it is a sign of the times.

About the XBox though, you may rightly complain that I cheated. I don’t think I did. For me, Tech Sabbath is about breaking my device-addiction. If I was addicted to playing games on my XBox, I would safely include it into the list. The way life has shaped itself for me over the last couple of years, I rarely if ever get time to play games–it is important to mention this, because I used to be an ardent, hardcore gamer. In other words, by not wiling time away glued to my devices, I played an immersive game on a big screen. The Far Cry franchise has always been renowned for its immersive gaming experience as well as the breathtaking free world environment it offers to its players. It’s immersive because when you play it, you feel less like playing a game than living through one. It’s an open world, and everything in it is as true to real life as can be.

I will confess though that I did, near the end of the day, pull up my laptop in bed not only to write this but also to read a foreboding article on how climate change is drastically going to afflict further the country in which I live. I have long, in my dreams as well as in my waking hours, worried over the state of growing natural calamities of late, the increasing lack of balance in nature and the ultimate impact it will all have on our world. It is macabre, judging from the way things are headed. The sign of the times that is hiding round the bend ahead. But that is a solemn topic for another day.

The small lifestyle change I’ve made.

Two years ago I made a lifestyle change by quitting my work-from home job, after committing four years of my life to it, and joining a full-time position at a physical office a half an hour drive away from where I live. I wrote about it, as well as other changes I brought about and embraced in my life, earlier here. For me, it was undoubtedly a big change.

Over the two years since, I felt that my lifestyle took on a downward bend. I was sleeping late, sleeping badly, waking up multiple times through the night. I was, as a result, getting out of bed late. With working parents leaving for work early, I was having the entire house to myself. I was lazing around the house, making my own breakfast–not quite the big deal everybody makes it, something I’ve been doing for a long while–watching TV in between, and getting ready to leave for work. I was coming to work late–which because of flexible timings at work was never a deterrent–and therefore leaving work late. Consequently, the lifestyle I had quietly slipped into was leaving me with no time to do anything else.

I accepted it as the way life was. Routine is just that. Once you settle into a routine, you accept it and refuse to believe there may be something wrong. That is how a major portion of your life is spent, following a routine day in, day out, unfalteringly. I accepted I had become a zombie and didn’t find any reason to complain. I saw no meaning in life. Ultimately, a zombielike routine for a life that held no meaning sounded perfectly alright to me.

I had the power to change it, but inertia lulled me to the dull satisfaction of that life. Why bother adjusting the cogs when they were moving mechanically in stride. Why bother risking bringing chaos into the otherwise imagined order and comfort of the system.

That was worrying. I was wrong. I had to do something.

And so, I started with the little steps. You hear about people dealing with addictions and habits either gradually or by cutting them off completely in one fell swoop. I felt, for me, the patient but determined approach was more likely to yield results.

Starting last month, I have been making a concerted push to change bits and pieces of my lifestyle. I wake up, and get out of bed, without fail between seven and half past seven in the morning–which is two to two and half hours earlier than what I had spoiled myself by allowing the luxury of. Whether sleepy, tired, regardless of how late I slept, I do it. Unfailingly.

I walk out for a wee bit, taking in the crisp and sometimes damp air. Notwithstanding how sultry the weather mercilessly is, the mornings are always brisk to some extent. And quiet.

I take my breakfast early too. The want of lazing time away has now been replaced by a need for doing things with some urgency. That gets me going.

Instead of getting into work around noon, I walk in around half past nine when the office is mostly empty, quiet and calm. No din of stand-alone fans, no annoying variable pitched voices chattering about. The overwhelming feeling of emptiness of space makes itself felt strongly. And the endearing quietness. For a month I have not failed to notice them. Not failed to appreciate them.

Instead of leaving work when the world was getting ready to put an end to their day, I leave work behind early at the end of the evening. I leave when the world outside is still naturally aglow and make it a point to appreciate it every evening. It allows me ample time to do what I will with it. I am home early to spend time with family. I hit the courts early to play when I feel like. I have sufficient time to go out with family for groceries and other activities that I apathetically shrugged away before.

Reading in the morning after coming to work is pleasant and distraction free. A fresh shower and a smooth drive to work, by avoiding taking chocked routes, do wonder to the mind. I feel brisk from the mornings till late afternoons. I can read more without loss of focus–if not the dwindling absence of it. I can write without friction. My mind actively takes on the day’s array of work.

It’s not all lovely though. I feel tired and devoid of sleep. I still don’t sleep well–but I believe that has deeper roots. I fall asleep easily because I’m too tired by the end of the day. My stark stance of not finding meaning in life hasn’t been affected, although I doubt very much that such a metaphysical complication can so much as be cured by a change in lifestyle.

It’s merely the beginning of it. That I feel the fruit of this gradual process, never minding the scars and the mud sticking on the surface of it, I feel motivated to push it further.