CloudTrail is a service provided by AWS that monitors all activity in your account, including API actions performed by IAM users. It is useful for performing security checks, but the default search console for it is not the largest.
Links CloudTrail and CloudWatch
By default, CloudTrail logs all events during the last 90 days in your account. But to link it to CloudWatch, you need to create a track that keeps track of events longer and also has the ability to keep extended logs of individual S3 writings and Lambda calls.
To create one, go to the CloudTrail Management Console and under the “Trails”; tab, create a new one. You can choose which regions and which types of events it monitors.
Under “Data events” you can also enable extended monitoring of S3 buckets or Lambda functions. These are optional and will incur additional costs, as well as take up much more storage space in CloudWatch Logs.
Once the track has been created, you can activate the CloudWatch Logs integration by clicking on the track name under “Trails”, scrolling down to “CloudWatch Logs” and pressing “Configure”.
The only option here is the log group name, by default
CloudTrail/DefaultLogGroup. The group is created if it does not already exist.
Because of how the AWS authentication system works, you need to give CloudTrail sufficient privileges to access CloudWatch log groups and create streams to start sending log events. This role is already configured, and all you have to do is tap “Allow” on the next screen to link the two services.
You should now see the log group and the IAM role under the track settings:
And in CloudWatch you will see a new log group and log stream created, which will start streaming all events automatically.
CloudWatch will receive all updates in the future, but at present there is no built-in way to import past events.
Links CloudWatch and Elasticsearch
Elasticsearch is a search engine commonly used to analyze Linux log files and is often linked to Kibana, a visualization engine that can draw graphs and plots using data from Elasticsearch. With the sheer amount of data that an active AWS account can spit from CloudTrail, Elasticsearch makes sense to many people. Fortunately, it’s pretty easy to set up.
You can read our complete guide to setting up an Elasticsearch server on AWS here. But if you’ve already configured it, it’s easy to link CloudWatch to it. Select the log group you want to link from the CloudWatch console and select “Stream to Amazon Elasticsearch Service”:
This opens a dialog where you can select your ES cluster. Then you should see all events from Elasticsearch.