This short story starts with an accidental discovery of how frequent Log4Shell attacks are executed against businesses and public internet services in general.
During my daily work at Dynatrace I come to test a lot of software services along with closely observing those through the eyes of a monitoring platform vendor.
I am running a test ‘hello world’ service on an AWS instance that is written in Go in order to run some standard software observability tests.
One of those standard tests is to observe a continuous service load and to learn a baseline for the response time of the service. Within that test scenario I am running 15 requests cron-job with a simple cURL cmd against the service endpoint every minute.
The Dynatrace observed service load then shows a constant 15 requests per minute and a steady response time.
While I was following those numbers I came to realize that Dynatrace shows a regular spike of around 400 requests every 24 hours at 1 am in the morning, as it is shown below:
As I am running artificial load, I was sure that those requests are not originating from my own cron-job.
When diving deeper within Dynatrace to see the individual traces, I realized that those requests are requesting all kinds of resources from my Go service. My ‘hello world’ Go service always responds with a HTTP 200 code, no matter which HTTP resource is requested, which leads to a collection of quite weird requests, as they are shown below:
All those requests that requested a cgi ressource were handled by my test service by just answering with a hello world message, which was probably not what the sender of those requests had in mind.
As Dynatrace allows me to inspect all the trace HTTP parameters, I found that the user agent http header field contained the answer to my question why those requests are sent in the first place.
The user agent header shows a weird script command that the log4shell exploit uses to attack a destination host. In case the destination host is running a vulnerable Java library version of log4j and if the web server tries to log the http request into a file, your server would be taken over by the attacker.
See below the header fields shown in Dynatrace trace observability that both shows the dangerous user agent field as well as the source IP address of the attacker.
Reporting the attack
I was quite stunned when I was told by our application security experts that those attacks are widespread. What also concerned me was that this attack is still ongoing and has been running for months on a daily interval, so nobody seems to care about it.
As I had all the IP addresses of the attackers (most likely those IPs are servers of victim companies as well who did not realize that they were hacked in the past) at hand, I decided to check who owns those addresses.
The address above is part of the Amazon AWS EC2 cloud offering, where I also found the right address to report the attack, which I did as shown below:
I also saw what Amazon advises people to collect, in order to be able to identify the attacker due to dynamic changes of IP addresses within their system.
I immediately got a response back to inform me that the report has been received and that the case will be investigated.
Lets see what comes out of this and if AWS is able to stop those regular attacks to my webserver.
