As the title says, I'm going to create my own blog software. Why, you ask? Well, I want to be able to write my blog posts in markdown, complete with code formatting and syntax highlighting. I've managed to get the syntax highlighting to work with blogger, but it requires hand editing HTML, which is annoying to say the least. There are other options, but most of them I either strongly dislike, or have a very long history of security issues. A lot of them also do not support markdown. So, why write this post about it? What's the point, you may ask. I am a huge proponent of not reinventing the wheel. And making the decision to write software is not to be taken lightly. My rule of thumb is that if something meets 90% of your needs, you should ask the following questions: 1. Can I live without the 10%? 2. Is there a way to extend the software to add part, or all of the 10%? 3. If there is a way, will it be so painful, that the investment to write your own is worthwhile? In this ...
Designing scalability, by being inefficient. You ain't gonna need it When I hear about dealing with things at scale, I often find people reach for batching of requests, caching, and other similar ideas. I'd like to challenge this thought process. Let's say you have a Movie's API, and users can get a list of movies, a single movie by ID, or many movies in a single request. What does this look like on the backend? Well, you could handle the batch request with a batch query to the data store. Seems efficient, right? And I'd argue it is. But what if it's 100 records, 1,000 records, 10,000 records? You'd probably want to distribute the load right? You could break that single request up into smaller batches, and query in parallel right. But what if, you send 1 HTTP request per record (with caching by ID)? Seems wasteful, right? System outline If you do that, however, you can get crafty. Let me give a small outline of the single obje...