AWS – cost optimization

AWS (Amazon Web Services) is the undisputed leader in the cloud technology market. The benefits of implementing solutions provided by AWS are spectacular, and the range of possibilities will satisfy even the most demanding users and administrators.

In my experience, however, I often come across this question – why do we need all the server administration in cloud solutions? After all, it’s enough to place the application, configure the server, and that’s it. There is nothing more to do there.

Well, that’s not true — people who ask this question usually come back to me after some time. Then I hear: “You set this app wrong! Why does it cost so much? Why do I pay more and more from month to month? Please take care of this immediately, this matter has the highest priority! “

The most likely reason behind this is the lack of proper server administration. Only then comes time for reflection – maybe seriously managing this infrastructure is worth it after all? Shouldn’t you start monitoring the processes taking place in it, anticipate potential costs, and adapt it to the current needs of the company?

In addition to the obvious advantages of global coverage, rapid innovation implementation, and compatibility with the agile management system, AWS offers great opportunities for cost optimization. Below are some tips to help you identify the reasons behind high costs and eliminate them effectively.

Considerable competition in the industry, a dynamically changing environment, and the emergence of new technologies is the everyday reality of IT specialists. The desire to lower this pressure, minimize costs, and reduce product development time often have a negative impact on the code quality. This is particularly evident in startups where carelessness is often confused with the Lean Startup approach. This, of course, has its advantages (quick generation of first revenue), but it’s also risky. Considering the matter from the long-term perspective, such deficiencies in the project may become a source of huge costs in the future.

An example would be the seemingly innocent idea: “Let’s put a database and application on one server without containerization.” It would seem that it’s a perfect solution. The database and the application can communicate with each other locally (and therefore quickly), and the configuration will not cause us difficulties. In addition, storing logs in the same database used by the application will facilitate accessing them. What about backups? Let’s do it locally and store them on the same server!

As soon as the application starts generating more traffic, the costs will automatically increase, and – in addition – such architecture will entail a number of other negative consequences. The database will become larger and slower, the disk capacity will start to run out, and the application will run much slower. Enlarging the server is also not a good solution, because its volume will be fully used only during the highest traffic volume.

It is definitely better to separate the database and backups from the application server. Dedicated AWS services are much cheaper than server instances. With this simple procedure, you can reduce the total cost of AWS by up to 50%.

The next step after separating the server application from the database and backups can be an overview of the size of our servers. Sometimes, for fear of too much load on the server, we add to it up to 50% more computing power than is actually needed. In this case, only 10 to 30% of the available computing power is used outside of the highest load. We pay for the power that is not used.

It’s worth checking what is causing this server load. This could be a thread that loops during backup, growing traffic on the application’s page, or a vector of attack on our application. Depending on the reason, you can do the appropriate correction in the application source code and permanently reduce the size of the instance.

If the reason for the CPU load is increasing website traffic, you can use the so-called Scaling Groups. This involves configuring the group so that at least one server instance is always available. After detecting the CPU load on this instance under certain conditions (e.g., above 80% for 3 minutes), another instance is added that receives some of the traffic. To save costs, you can set an instance limit, e.g., a maximum of 5.

When the CPU on additional instances drops below 20%, the Scaling Group will automatically remove the additional instance. That way, we will only bear the costs when the increased computing power is actually needed. Depending on the cause and architecture used, our savings can reach up to 70% of the total AWS costs.

If the instance size is constant, it will be used for another year, and we do not expect changes to the application at this instance, it’s worth using the instance reservation. A one-time payment for the entire year reduces costs by 15%.

Nowadays, users are very demanding. Too long loading time, slow data transfer, or stuttering progress bar can cause a lot of frustration. However, is speedy access to data necessary in every situation?

Slower resources are cheaper. Currently, most applications collect huge amounts of logs. Depending on their purpose and frequency of use, they differ from each other in size, form, location, and form of storage, structure, or access method.

Logs can be stored in a relational database, in a text file, in Elasticsearch, etc. It’s worth considering whether all logs must be quickly available. Could logs older than three weeks be available on-demand, e.g., only after two days? If so, then it’s worth creating a slower but cheaper resource for storing older logs and performing a log recovery test. This solution allows you to reduce costs by up to 70%!

Poorly chosen architecture, improper infrastructure management, and underestimating the role of server administrators can generate high costs. Do you want to find out how to best optimize costs and start to use the potential of AWS services fully? Contact us – we will answer all your questions, help you choose and implement solutions that will facilitate your work and take your business to the next level.