Fluency AWS Monitoring
Sections
AWS Cloudwatch Metrics
A key component of monitoring (and observability) is the collection and processing of metrics. In the AWS ecosystem, these metrics are found in AWS Cloudwatch Metrics.
CloudWatch Metrics are obtained through API polling, or the newer Metric Streams.
Fluency integration concepts
Fluency’s integration will use the AssumeRole API to securely connect with your intermediary resources without the need to pass around IAM credentials. An IAM Policy w/ External ID will explicitly define Fluency’s scope of access to your account.
At any time, you may remove the integration and/or intermediary resources to revoke access.
https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html
The integration consists of three sections:
-
The user will first initiate the integration on the Fluency Portal. Creating a new plugin from the Integration section will generate a unique External ID for use in the client AWS account.
-
Fluency provides AWS CloudFormation scripts for the user to create AWS resources for the integration. Executing the CloudFormation with the External ID, will create AWS resources.
-
The final step will test the above AWS configuration in the Fluency interface. Once the test is successful, the user can complete the rest of the integration.
Fluency Web Interface (Part 1)
Login to the Fluency Cloud portal: https://companyname.cloud.fluencysecurity.com.
Open the main drop-down menu and choose the Cloud Integrations option under the Data Ingress section.
On the following page, navigate to the Cloud Infrastructure as a Service section.
To Add an integration for AWS Monitoring, choose the AWS icon from the group on the left side of the page to create a new AWS integration endpoint.
NOTE: If an integration endpoint was setup previously, you can also select and modify it from the right side of the page.
In the pop-up window, provide an Account name for the integration. The value will be used within the Fluency interface only to distinguish the different integrations. It is suggested to avoid using spaces in this field.
Also, enter the AWS Account ID for your account. Choose the Save button to add the integration endpoint.
For AWS GovCloud support, check the GovCloud box
Select the AWS integration endpoint from the list on the right side of the page, in the same Cloud Infrastructure as a Service section. Choose the pencil icon to edit/configure the connector.
For the initial configuration, this will bring up a pop-up window, with the Account name and the External ID.
The External ID will be required in the following step on the AWS console.
After executing the CloudFormation script, you are given the opportunity to test the configuration using the Test Connection button.
On subsequent visits for the plugin configuration page, select the Show External ID button in the upper left corner of the page.
AWS CloudFormation
AWS CloudFormation allows you to configure AWS resources from script/code. This makes deployment easy, and consistent, and greatly decrease the possiblity of errors or mis-configurations.
The following resources will be created on AWS:
- 1 IAM Role
- 2 IAM Policy
While the script is free to use, keep in mind that AWS CloudFormation is a paid service, and you will incur a charge from AWS for using it.
Link to the CloudFormation file on S3 (initialization script):
https://fluency-cloudformation.s3.us-east-2.amazonaws.com/FluencyAWSMonitoringV4.yaml
Link to the CloudFormation file on S3 (initialization script for AWS GovCloud users):
https://fluency-cloudformation.s3.us-east-2.amazonaws.com/FluencyAWSMonitoringGovCloud.yaml
Deploying a CloudFormation template
Navigate to the CloudFormation section of the AWS Management Console.
Under the “Stacks” section, choose “Create stack” (with new resources, standard).
On the following page, specify a template, and choose the desired template (using Amazon S3 URL) from above:
Click Next to continue.
Give this deployment a name, and specify some parameters. You can choose your own name, or keep the default values provided by Fluency.
Click Next to continue.
Configure additional items (optional).
Click Next to continue.
Review the deployment, when complete, choose Create stack to deploy.
The deployment in progress
When the deployment is complete, return to the Fluency interface for the following step.
Fluency Web Interface (Part 2)
Return the Integrations page, and edit the AWS Monitoring configurations.
On the following pop-up, you are given the opprtunity to test the configuration using the Test Connection button.
Proceed to the next page, if the test completes successfully. (This is also your last chance to view the ExternalID.)
You may be sent to the edit page directly, on subsequent visits for the plugin configuration page, select the External ID tab in the upper left corner of the page.
Adding a Region
Navigate to the plugin configuration page, select the + Metric Poll button in the Cloudwatch Regions tab to integrate / enable monitoring in a specific AWS Region of your account.
Add Metric Poll Wizard
In the pop-up window, select the appropriate AWS Region to enable. Available Regions are obtained via an API call to your AWS account.
Select the desired Region, and click the Save button.
A dialog will confirm the successful region addition.
This completes the initial integration.
Adding Metric Streams
AWS CloudWatch Metric Streams replaces traditional polling methods to deliver large quantities of data directly to an AWS S3 bucket.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Metric-Streams.html
Integration concepts
Fluency’s integration with AWS Cloudwatch is capable of collecting events from Metric Streams, via an intermediary Amazon Kinesis Data Firehose and AWS S3 bucket. Our integration will also deploy AWS SQS and IAM resources to facilitate data collection.
Metric Stream support is now available through the interface. However this feature will require a different CloudFormation script, and additional instructions.
Fluency provides a CloudFormation script to initialize the integration, and add one (1) Metric stream integration, via an AWS SQS queue. Fluency also provides another “update” CloudFormation script to add additional metric streams after the first one
Please contact Fluency Support if you would like to configure Metric Stream support.
Page last updated: 2022 Feb 22