At your company, developers can spin up EC2 instances as needed. As the platform team, you want to ensure proper observability by automatically deploying OpenTelemetry collectors to gather metrics and provide standardized monitoring endpoints.
Cloud computing platform that provides scalable compute capacity. We’ll use EC2 instances as our deployment targets for running applications and collecting metrics.
A vendor-agnostic way to receive, process and export telemetry data. The collector will be automatically deployed to gather system metrics and provide a standardized endpoint for custom application metrics.
CI/CD platform that will automate our deployment pipeline. It will handle the provisioning of collectors and configuration management across all environments.
Before deploying OpenTelemetry collectors, EC2 instances must be registered as Resources in Ctrlplane. These Resources represent the external state of our EC2 instances and require consistent updates as the state of each instance changes.
These are just some of the ways you can get EC2 instances into Ctrlplane.
Using AWS Lambda
AWS Lambda can be used to automatically register EC2 instances in Ctrlplane whenever they are launched. By setting up a Lambda function triggered by CloudWatch Events, you can capture instance launch events and use the Ctrlplane API to create or update resources representing these instances.
Using AWS Systems Manager
AWS Systems Manager provides a way to manage EC2 instances at scale. By integrating Systems Manager with Ctrlplane, you can automatically register instances as they are launched. This can be achieved by using Systems Manager Run Command to execute scripts that call the Ctrlplane API for resource registration.
Using Ctrlplane CLI
AWS CloudFormation can be used to define and provision EC2 instances as well as register them in Ctrlplane. By including a custom resource in your CloudFormation template, you can trigger a Lambda-backed custom resource that registers the instance in Ctrlplane upon creation.
In this tutorial, we’ll simplify the process by using the Ctrlplane CLI to manage the state of EC2 instances within Ctrlplane. This approach keeps our deployment pipeline straightforward and concentrated on the primary task. However, a limitation is that the pipeline executes every 10 minutes. Consequently, if a new EC2 instance is launched, it won’t trigger a deployment of OTEL until the next 10-minute cycle.
While this example focuses on AWS EC2, the solution can be applied to any resource type. As long as you can create resources representing your deployment target in Ctrlplane, you can automate the deployment of OpenTelemetry collectors and provide standardized monitoring endpoints for your development teams.