ENG-4941 Enable high-resolution ECS scaling metrics#68
Draft
mabadir wants to merge 2 commits into
Draft
Conversation
Ravion Module Publish PlanDry run only. No Ravion API mutations were made.
Diffsrvn-ecs-nlb 0.2.2 -> 1.0.0--- remote
+++ compiled
show_when:
auto_scaling_enabled: true
type: number
+ - collapsible: true
+ default: true
+ description: Publish CPU and enabled memory metrics every 20 seconds so target tracking can respond faster. High-resolution metrics add CloudWatch charges.
+ id: high_resolution_metrics_enabled
+ label: High-resolution scaling metrics
+ show_when:
+ auto_scaling_enabled: true
+ type: boolean
- default: 1
description: Number of tasks to keep running when autoscaling is disabled.
id: desired_count
@@
The module is intentionally focused on Layer 4 services behind a Network Load Balancer. Use ECS Web Service for HTTP host and path routing through an Application Load Balancer.
- Terraform source: [flightcontrolhq/modules/compute/ecs_service](https://github.com/flightcontrolhq/modules/tree/rvn-ecs-nlb@0.2.2/compute/ecs_service)
+ Terraform source: [flightcontrolhq/modules/compute/ecs_service](https://github.com/flightcontrolhq/modules/tree/rvn-ecs-nlb@1.0.0/compute/ecs_service)
## Use cases
@@
## Autoscaling
- Autoscaling is enabled by default with one minimum task and three maximum tasks. For production, start with at least two minimum tasks when availability matters. When autoscaling is disabled, Desired tasks controls how many tasks the ECS service keeps running.
+ Autoscaling is enabled by default with one minimum task and three maximum tasks. High-resolution scaling metrics are also enabled by default, publishing the built-in CPU and configured memory metrics every 20 seconds so target tracking can react faster; this adds CloudWatch charges. Turn off High-resolution scaling metrics to use the standard 60-second policies. For production, start with at least two minimum tasks when availability matters. When autoscaling is disabled, Desired tasks controls how many tasks the ECS service keeps running.
| Field | Default | Description |
| ------------------------- | ------- | --------------------------------------------- |
@@
| Minimum tasks | 1 | Lower bound for running tasks |
| Maximum tasks | 3 | Upper bound for running tasks |
| Desired tasks | 1 | Running tasks when autoscaling is disabled |
+ | High-resolution scaling metrics | true | Uses 20-second CPU and memory scaling metrics |
| CPU target (%) | 70 | Average CPU utilization target |
| Memory target (%) | 80 | Average memory utilization target |
| Scale-in cooldown (secs) | 300 | Delay before another scale-in action |
@@
| CPU architecture | No | X86_64 | x86_64 compatibility or ARM64 cost optimization |
| ECS exec | No | false | Enable ECS Exec for debugging containers |
| Autoscaling | No | true | Enable CPU and optional memory target tracking |
+ | High-resolution scaling metrics | No | true | Use 20-second CPU and memory scaling metrics |
| Desired tasks | Yes* | 1 | Number of tasks when autoscaling is disabled |
| Deployment strategy | Yes | rolling | Rolling, Blue/green, Linear, or Canary |
| FireLens log routing | No | false | Route app logs through a Fluent Bit sidecar |
@@
base_path: compute/ecs_service
branch: main
execution_environment_id: << module.input.execution_environment_id >>
- ref: rvn-ecs-nlb@0.2.2
+ ref: rvn-ecs-nlb@1.0.0
repo: https://github.com/flightcontrolhq/modules
stack_id: <<stack.id>>
terraform_variables:
@@
scheduled: "<< module.input.scheduled_scaling != nil ? module.input.scheduled_scaling : [] >>"
target_tracking:
- policy_name: cpu
- predefined_metric: ECSServiceAverageCPUUtilization
+ predefined_metric: '<< module.input.high_resolution_metrics_enabled ? "ECSServiceAverageCPUUtilizationHighResolution" : "ECSServiceAverageCPUUtilization" >>'
scale_in_cooldown: << module.input.scale_in_cooldown >>
scale_in_enabled: << module.input.scale_in_enabled >>
scale_out_cooldown: << module.input.scale_out_cooldown >>
target_value: << module.input.cpu_target_value >>
- - '...<< module.input.memory_target_value ? [{policy_name: "memory", predefined_metric: "ECSServiceAverageMemoryUtilization", target_value: module.input.memory_target_value, scale_in_cooldown: module.input.scale_in_cooldown, scale_out_cooldown: module.input.scale_out_cooldown, scale_in_enabled: module.input.scale_in_enabled}] : [] >>'
+ - '...<< module.input.memory_target_value ? [{policy_name: "memory", predefined_metric: (module.input.high_resolution_metrics_enabled ? "ECSServiceAverageMemoryUtilizationHighResolution" : "ECSServiceAverageMemoryUtilization"), target_value: module.input.memory_target_value, scale_in_cooldown: module.input.scale_in_cooldown, scale_out_cooldown: module.input.scale_out_cooldown, scale_in_enabled: module.input.scale_in_enabled}] : [] >>'
- "...<< module.input.custom_metric_scaling_policies != nil ? module.input.custom_metric_scaling_policies : [] >>"
capacity_provider_strategies:
- '...<< module.input.capacity_provider == "fargate" || module.input.additional_fargate_capacity_enabled ? [{capacity_provider: module.input.fargate_capacity_provider_name, weight: 1, base: 0}] : [] >>'rvn-ecs-web 0.8.2 -> 1.0.0--- remote
+++ compiled
show_when:
auto_scaling_enabled: true
type: number
+ - collapsible: true
+ default: true
+ description: Publish CPU and enabled memory metrics every 20 seconds so target tracking can respond faster. High-resolution metrics add CloudWatch charges.
+ id: high_resolution_metrics_enabled
+ label: High-resolution scaling metrics
+ show_when:
+ auto_scaling_enabled: true
+ type: boolean
- default: 1
description: Number of tasks to keep running when autoscaling is disabled.
id: desired_count
@@
The module is intentionally focused on web services behind an Application Load Balancer. It uses the selected ECS cluster to inherit AWS account, region, VPC, subnets, capacity providers, load balancer listeners, and load balancer security groups.
- Terraform source: [flightcontrolhq/modules/compute/ecs_service](https://github.com/flightcontrolhq/modules/tree/rvn-ecs-web@0.8.2/compute/ecs_service)
+ Terraform source: [flightcontrolhq/modules/compute/ecs_service](https://github.com/flightcontrolhq/modules/tree/rvn-ecs-web@1.0.0/compute/ecs_service)
## Use cases
@@
## Autoscaling
- Autoscaling is enabled by default with one minimum task and three maximum tasks. For production, start with at least two minimum tasks when availability matters. When autoscaling is disabled, Desired tasks controls how many web tasks the ECS service keeps running.
+ Autoscaling is enabled by default with one minimum task and three maximum tasks. High-resolution scaling metrics are also enabled by default, publishing the built-in CPU and configured memory metrics every 20 seconds so target tracking can react faster; this adds CloudWatch charges. Turn off High-resolution scaling metrics to use the standard 60-second policies. For production, start with at least two minimum tasks when availability matters. When autoscaling is disabled, Desired tasks controls how many web tasks the ECS service keeps running.
| Field | Default | Description |
| ------------------------ | ------- | ---------------------------------------------- |
@@
| Minimum tasks | 1 | Lower bound for running tasks |
| Maximum tasks | 3 | Upper bound for running tasks |
| Desired tasks | 1 | Running tasks when autoscaling is disabled |
+ | High-resolution scaling metrics | true | Uses 20-second CPU and memory scaling metrics |
| CPU target (%) | 70 | Average CPU utilization target |
| Memory target (%) | 80 | Average memory utilization target |
| Scale-in cooldown (secs) | 300 | Delay before another scale-in action |
@@
| Post-deploy ephemeral storage | No | Task definition default | Ephemeral storage override for the post-deploy task |
| Post-deploy timeout (secs) | No | 1800 | Maximum post-deploy task wait time |
| Autoscaling | No | true | Enable CPU and optional memory target tracking |
+ | High-resolution scaling metrics | No | true | Use 20-second CPU and memory scaling metrics |
| Desired tasks | Yes* | 1 | Number of web tasks when autoscaling is disabled |
| Deployment strategy | Yes | rolling | Rolling, Blue/green, Linear, or Canary |
| EFS file system | No | false | Mount an EFS file system into the app container |
@@
base_path: compute/ecs_service
branch: main
execution_environment_id: << module.input.execution_environment_id >>
- ref: rvn-ecs-web@0.8.2
+ ref: rvn-ecs-web@1.0.0
repo: https://github.com/flightcontrolhq/modules
stack_id: <<stack.id>>
terraform_variables:
@@
scheduled: "<< module.input.scheduled_scaling != nil ? module.input.scheduled_scaling : [] >>"
target_tracking:
- policy_name: cpu
- predefined_metric: ECSServiceAverageCPUUtilization
+ predefined_metric: '<< module.input.high_resolution_metrics_enabled ? "ECSServiceAverageCPUUtilizationHighResolution" : "ECSServiceAverageCPUUtilization" >>'
scale_in_cooldown: << module.input.scale_in_cooldown >>
scale_in_enabled: << module.input.scale_in_enabled >>
scale_out_cooldown: << module.input.scale_out_cooldown >>
target_value: << module.input.cpu_target_value >>
- - '...<< module.input.memory_target_value ? [{policy_name: "memory", predefined_metric: "ECSServiceAverageMemoryUtilization", target_value: module.input.memory_target_value, scale_in_cooldown: module.input.scale_in_cooldown, scale_out_cooldown: module.input.scale_out_cooldown, scale_in_enabled: module.input.scale_in_enabled}] : [] >>'
+ - '...<< module.input.memory_target_value ? [{policy_name: "memory", predefined_metric: (module.input.high_resolution_metrics_enabled ? "ECSServiceAverageMemoryUtilizationHighResolution" : "ECSServiceAverageMemoryUtilization"), target_value: module.input.memory_target_value, scale_in_cooldown: module.input.scale_in_cooldown, scale_out_cooldown: module.input.scale_out_cooldown, scale_in_enabled: module.input.scale_in_enabled}] : [] >>'
- "...<< module.input.custom_metric_scaling_policies != nil ? module.input.custom_metric_scaling_policies : [] >>"
capacity_provider_strategies:
- '...<< module.input.capacity_provider == "fargate" || module.input.additional_fargate_capacity_enabled ? [{capacity_provider: module.input.fargate_capacity_provider_name, weight: 1, base: 0}] : [] >>'rvn-ecs-worker 0.3.2 -> 1.0.0--- remote
+++ compiled
show_when:
auto_scaling_enabled: true
type: number
+ - collapsible: true
+ default: true
+ description: Publish CPU and enabled memory metrics every 20 seconds so target tracking can respond faster. High-resolution metrics add CloudWatch charges.
+ id: high_resolution_metrics_enabled
+ label: High-resolution scaling metrics
+ show_when:
+ auto_scaling_enabled: true
+ type: boolean
- default: 1
description: Number of tasks to keep running when autoscaling is disabled.
id: desired_count
@@
The ECS Worker module creates an ECS service for background jobs, queue consumers, event processors, and other private workloads in an existing Ravion ECS cluster. It uses the same ECS service Terraform module as ECS Web Server, but does not create or attach a load balancer target group and does not expose a primary container port.
- Terraform source: [flightcontrolhq/modules/compute/ecs_service](https://github.com/flightcontrolhq/modules/tree/rvn-ecs-worker@0.3.2/compute/ecs_service)
+ Terraform source: [flightcontrolhq/modules/compute/ecs_service](https://github.com/flightcontrolhq/modules/tree/rvn-ecs-worker@1.0.0/compute/ecs_service)
## Use cases
@@
## Autoscaling
- Autoscaling is enabled by default with CPU and memory target tracking. Use custom metric scaling policies for queue depth, lag, backlog, or other workload-specific metrics that better represent worker demand. When autoscaling is disabled, Desired tasks controls how many worker tasks the ECS service keeps running.
+ Autoscaling is enabled by default with CPU and memory target tracking. High-resolution scaling metrics are also enabled by default, publishing both built-in metrics every 20 seconds so target tracking can react faster; this adds CloudWatch charges. Turn off High-resolution scaling metrics to use the standard 60-second policies. Use custom metric scaling policies for queue depth, lag, backlog, or other workload-specific metrics that better represent worker demand. When autoscaling is disabled, Desired tasks controls how many worker tasks the ECS service keeps running.
## Configuration
@@
| App size | No | 2 vCPU, 4 GB | Fargate task CPU and memory |
| App ephemeral storage (GiB) | No | 20 | Fargate task ephemeral storage |
| Autoscaling | No | true | Enable CPU and optional memory target tracking |
+ | High-resolution scaling metrics | No | true | Use 20-second CPU and memory scaling metrics |
| Desired tasks | Yes* | 1 | Number of worker tasks when autoscaling is disabled |
| Custom metric scaling policies | No | [] | Queue/backlog-oriented target tracking policies |
| Runtime environment variables | No | - | Environment variables passed to the worker container |
@@
base_path: compute/ecs_service
branch: main
execution_environment_id: << module.input.execution_environment_id >>
- ref: rvn-ecs-worker@0.3.2
+ ref: rvn-ecs-worker@1.0.0
repo: https://github.com/flightcontrolhq/modules
stack_id: <<stack.id>>
terraform_variables:
@@
scheduled: "<< module.input.scheduled_scaling != nil ? module.input.scheduled_scaling : [] >>"
target_tracking:
- policy_name: cpu
- predefined_metric: ECSServiceAverageCPUUtilization
+ predefined_metric: '<< module.input.high_resolution_metrics_enabled ? "ECSServiceAverageCPUUtilizationHighResolution" : "ECSServiceAverageCPUUtilization" >>'
scale_in_cooldown: << module.input.scale_in_cooldown >>
scale_in_enabled: << module.input.scale_in_enabled >>
scale_out_cooldown: << module.input.scale_out_cooldown >>
target_value: << module.input.cpu_target_value >>
- - '...<< module.input.memory_target_value ? [{policy_name: "memory", predefined_metric: "ECSServiceAverageMemoryUtilization", target_value: module.input.memory_target_value, scale_in_cooldown: module.input.scale_in_cooldown, scale_out_cooldown: module.input.scale_out_cooldown, scale_in_enabled: module.input.scale_in_enabled}] : [] >>'
+ - '...<< module.input.memory_target_value ? [{policy_name: "memory", predefined_metric: (module.input.high_resolution_metrics_enabled ? "ECSServiceAverageMemoryUtilizationHighResolution" : "ECSServiceAverageMemoryUtilization"), target_value: module.input.memory_target_value, scale_in_cooldown: module.input.scale_in_cooldown, scale_out_cooldown: module.input.scale_out_cooldown, scale_in_enabled: module.input.scale_in_enabled}] : [] >>'
- "...<< module.input.custom_metric_scaling_policies != nil ? module.input.custom_metric_scaling_policies : [] >>"
capacity_provider_strategies:
- '...<< module.input.capacity_provider == "fargate" || module.input.additional_fargate_capacity_enabled ? [{capacity_provider: module.input.fargate_capacity_provider_name, weight: 1, base: 0}] : [] >>' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
High-resolution scaling metricstoggle to the shared Ravion autoscaling form, enabled by default1.0.0because the new default changes behavior and CloudWatch costPending upstream provider support
This PR is pending hashicorp/terraform-provider-aws#48792, which adds the
aws_ecs_service.monitoringschema and service-revision read path. Do not merge this PR until that change is merged and included in a provider release.Once the compatible provider release exists, this PR still needs:
tofu validateandtofu testrerun against that providerecs:DescribeServiceRevisionsNo CLI workaround is included.
Validation
tofu fmt -check— passedgit diff --check— passedrvn-ecs-web,rvn-ecs-nlb, andrvn-ecs-worker— attempted; blocked because the local Ravion API was not running atlocalhost:8080Linear: ENG-4941