I fear I won't work in technology again

So, last Friday, after a frantic few months of searching, I scored an onsite interview at last - my second one this year. I had reasons to be hopeful about it. I knew the recruiter who contacted me, from my preceding job when she was a social media herder, and the VP for the data department turned out to be a well-known name outside of work whom I knew at least by sight. I was aware I'd done very well on the coding challenge which featured a nontrivial SQL puzzle and a Python part where I corrected a design anti-pattern in the requirements (I don't know if it was an intentional trick question).

But it was all for nought. Right from the first session, it started surfacing that they are in fact running full speed away from SQL, except perhaps as a playground for data scientists. They were looking for someone to help building a data pipeline, starting with CSV data obtained from partners, transforming it into JSON, rotating the JSON in various ways, and finally loading the results into a huge Redis cluster. I tried to keep positive but my distaste must have been noticed - I had used Redis in the previous job and I know it is to SQL as Perl is to ML.

The rotating part was going to be implemented as microservices, and I was asked if I had written any. My reaction was, what does the term mean? Sure, I have written a couple small daemons talking lightweight protocols. But I have done that in C (before I even knew Python), and the protocols were line oriented, not JSON over HTTP. Does this disqualify me, especially when the JSON and HTTP part will all be done by mature 3rd party code?

And then there is Docker. By the time I was asked if I'd had any experience with it, I only managed a straight "no" answer, and didn't bother to mention my time working with VMware and Virtualbox.

Why is all this happening, where are the nice tools going to which I have a sort of emotional or aesthetic attachment, and which are backed by some theory? To the extent this is a technical question (and I want to only write about the technical part in this post), it's quite clear to me that the answer is size, otherwise known as scale. I believe this team (and others who tell or write the same) that under their growth assumptions a PostgreSQL database or even a cluster of them, running in the cloud, couldn't manage.

What's worse for me is that this is now the only kind of shop there is. No companies are looking for engineers to take care of their computing in house, they're all happy to let Google or Amazon handle that, or at any rate a startup whose goal is to become a Google or Amazon, if needed by getting acquired by one of them.

Oops, I'm veering into the non-technical. I want to reserve that for my other post or posts about my current adventures, perhaps in another space, which you may read soon. But please, if you know of an opening for someone with my old-fashioned skills, let me know soon. You can see my resume online.

Comments !

You can send comments on this post by email. Click here to start composing a comment.