The other day, I was talking to a Database Administrator about database fault tolerance and the different types of Transaction Logs. For reference in this article, a transaction log (also transaction journal or database log) keeps a history of actions (events) executed by a database management system used to guarantee ACID properties over crashes or hardware failures.
Databases store events internally to guarantee ACID properties. Does it mean a Database system implements an event driven architecture? The Short answer is no. And this is why.
This goes back to one of the greatest thought-provoking presentations from Martin Fowler about Event…
Guidelines for modern cloud-native architectures
AWS started in 2006. At that stage, it was almost unthinkable to foresee what we are going through today. Cloud was thought for start-ups and not ready for Enterprise. It took several years to become mainstream.
Today, that has changed. Companies are writing new software on the cloud and migrating legacy applications. Most likely, if your company is less than 10 years old, then you may not even have a physical server.
Software created to run on the cloud is sometimes referred to as cloud-native. However, does it mean that every application running on the…
The COVID-19 Leadership Guide about strategies for managing through this crisis [1] presents valuable insights in retrospect. There are several relevant topics, however, utilising existing IT assets or available technologies and tools caught my eye.
In many cases I have seen the first reaction to any problem is to buy something new. This happens almost instantaneously without taking into account related costs for migration, decommissioning, required development or customisation, impacted human resources, training and so on.
This also applies to available capabilities within current technologies and platforms. …
In accordance with the physical laws of gravity, objects with larger mass attract those with less. In this analogy, bigger data sets will pull closer applications, services, and smaller data sets due to latency and throughput.
Why is this concept important?
Data gravity is an interesting analogy because it is easier to move to compute around (applications and/or services) rather than moving petabytes between data centers. Let alone if it requires a data migration from one format to another. When planning the data center strategy, this needs to be considered.
As business data continues to become an ever-increasing commodity, it is essential that data gravity be taken into consideration when designing solutions that will use that data. One must consider not only current data gravity but its potential growth. Data gravity will only increase over time, and in turn, will attract…
Storing vast amounts of data comes with its own risks and challenges. Synthetic data is one of the options in the toolkit to address them. This article reviews different angles, including data confidentiality, retention and de-identification.
Census data is unreliable. There are several reasons; missing data, classification difficulties, erroneous, misreported data, among others. On top of these issues, regulatory requirements enforce to preserve anonymity when analysing the dataset.
In 1993 Donald Rubin, author of the book Statistical Analysis with Missing Data, had the original idea of fully synthetic data for privacy-preserving statistical analysis. He originally designed this to synthesize the Census long form responses for the short form households. He then released samples that did not include any actual long form records - in this he preserved anonymity of the household. Later that year, the idea…
Sending people to a training course with tea and cookies will not achieve the desired cultural change
I remember it clearly. It was almost 20 years ago and it was difficult. We wanted change, to improve our software development practice and release faster with less bugs. However, THEY didn’t want it. They wanted to keep building locally, handcraft the artefacts and release manually. For several months, a continuous disagreement that slowed and frustrate us all.
Polarisation or a division into two sharply contrasting groups will not make things better. And it will not make a team work as a single unit which is a basic ingredient for efficient teams. It could be worse, but not working in…
“API-first design” will pay off in the long term
SpaceX docks the Crew Dragon spaceship to the International Space Station after a 19-hours journey since it lifted off from NASA’s Kennedy Space Center in Florida. Great achievement as a result of lots of smart work. Imagine getting all the way up there to realise the interface has changed and it won’t dock. It would be too late to blame requirements, design, dev and/or test. And another iteration would come with an expensive price tag.
IP requires IC and SC, if IC goes low, then IP is almost lost. You may ask, Pablo, what are you talking about? I decided kick-off with the conclusion. So lets rollback and start from the beginning.
In copyright legislation around the world, source code is considered the Intellectual Property of the creator. Source code is protected in the same way as a “literary work”, which means it is “copyrightable” from the moment that the first line of code is created. [2]
When we start a project, we are full of ideas that (hopefully) get translated into source code that becomes a working system. In order to understand what our source code is doing and the business domain problem it is resolving, we use reasoning, and we have control using our Intellectual Control (IC).
In order to…
We have automated them, it is time to automate us
For the last 50 years, IT has been automating business functions; accounting, human resources, finance, rating, scoring, claims, among many others. The first decades, Software Engineering was largely manual with long cycles of development, manual testing and integration, especially integration. Followed by risky snowflake deployments. The definition of feature completion has been evolving over time [1].
In the last decades, we have been paying a lot more attention to automating the process of business automation. Software Engineering has been improving over time and productivity is higher than ever before. This trend has 3 major factors.
Frictionless development pipelines
Removing friction from the development…
You have to know the past to understand the present — Carl Sagan
Internet pace is frenetic. In a couple of months, articles and books become history. Slowing down and reviewing the past can be a rewarding experience, so I reflected on my notes and came up with the best articles of the last decade.
I have decided to order them chronologically so we spare the “which one is the best one” controversy and focus on content. I have left many outside the list and for sure there are a lot more. Please share your best ones in the comments.
Some of the greatest writers in this field are Martin Fowler, Joel Spolsky…
Pablo Iorio
Solutions Architect. Passionate technologist who believes in science and technology as a contributing factor for progress and prosperity.
