X Ray is a service that collects data from the requests that the application generates and further on top of that it filters, view and gain insights to identify issues and opportunities for optimization.

AWS services that are integrated with X-Ray can add tracing headers to incoming requests, send trace data to X-Ray, or run the X-Ray daemon. AWS X Ray receives data from services as segments and further it group segments into traces. X Ray process the traces to generate a service graph.

• Segments: The compute resources running your application logic send data about their work as segments. Segments provides resource name, details about the request, response, work done.
• Subsegments: For services that don’t send their own segments, like Amazon DynamoDB, X-Ray uses subsegments to generate inferred segments and downstream nodes on the trace map.
• A trace collects all the segments generated by a single request. That request is typically an HTTP GET or POST request that travels through a load balancer, hits your application code, and generates downstream calls to other AWS services or external web APIs. 
• A service graph is a JSON document that contains information about the services and resources that make up your application.
• Sampling: To ensure efficient tracing and provide a representative sample of the requests that your application serves, the X-Ray SDK applies a sampling algorithm to determine which requests get traced. 
• X-Ray indexes up to 50 annotations per trace. Annotations are simple key-value pairs that are indexed for use with filter expressions. Metadata are key-value pairs with values of any type, including objects and lists, but that are not indexed.
• Application signals: It allows to discover and monitor your application services, clients, Synthetics canaries, and service dependencies.
• The primary X-Ray console page is the trace map, which is a visual representation of the JSON service graph that X-Ray generates from the trace data generated by your applications. 
• Use filter expressions to view a trace map or traces for a specific request, service, connection between two services (an edge), or requests that satisfy a condition. 
• AWS X-Ray supports tracing event-driven applications using Amazon SQS and AWS Lambda.
• Groups are a collection of traces that are defined by a filter expression. You can now configure X-Ray groups from within the Amazon CloudWatch console. You can also continue to use the X-Ray console.

Instrumenting your application involves sending trace data for incoming and outbound requests and other events within your application, along with metadata about each request.

• Auto instrumentation – instrument your application with zero code changes, typically via configuration changes.
• Library instrumentation – make minimal application code changes to add pre-built instrumentation targeting specific libraries or frameworks, such as the AWS SDK, Apache HTTP clients, or SQL clients.
• Manual instrumentation – add instrumentation code to your application at each location where you want to send trace information.

What is Trace Map in AWS X Ray

The trace map is a visual representation of the trace data that’s generated by your applications.

The map shows service nodes that serve requests, upstream client nodes that represent the origins of the requests, and downstream service nodes that represent web services and resources that are used by an application while processing a request.

• Use the Traces page in the X-Ray console to find traces by URL, response code, or other data from the trace summary. 

What is AWS X-Ray Daemon?

The AWS X-Ray daemon is a software application that listens for traffic on UDP port 2000, gathers raw segment data, and relays it to the AWS X-Ray API. 

Instead of sending trace data directly to X-Ray, each client SDK sends JSON segment documents to a daemon process listening for UDP traffic. The X-Ray daemon buffers segments in a queue and uploads them to X-Ray in batches. The daemon is available for Linux, Windows, and macOS, and is included on AWS Elastic Beanstalk and AWS Lambda platforms.

The X-Ray daemon uses the AWS SDK to upload trace data to X-Ray, and it needs AWS credentials with permission to do that. On Amazon EC2, the daemon uses the instance’s instance profile role automatically.

How to enable the X-Ray daemon in the Elastic Beanstalk console.

1. Open the Elastic Beanstalk console.
2. Navigate to the management console for your environment.
3. Choose Configuration.
4. Choose Software Settings.
5. For X-Ray daemon, choose Enabled.
6. Choose Apply.

How to install AWS X Ray Daemon on EC2 instance.

#!/bin/bash
curl https://s3.us-east-2.amazonaws.com/aws-xray-assets.us-east-2/xray-daemon/aws-xray-daemon-3.x.rpm -o /home/ec2-user/xray.rpm
yum install -y /home/ec2-user/xray.rpm

To get started, the only option that you need to know is -n or --region, which you use to set the region that the daemon uses to send trace data to X-Ray.

~/xray-daemon$ ./xray -n us-east-2

How to Run the X-Ray daemon on Amazon ECS

In Amazon ECS, create a Docker image that runs the X-Ray daemon, upload it to a Docker image repository, and then deploy it to your Amazon ECS cluster. You can use port mappings and network mode settings in your task definition file to allow your application to communicate with the daemon container.

Example Task definition

    {
      "name": "xray-daemon",
      "image": "amazon/aws-xray-daemon",
      "cpu": 32,
      "memoryReservation": 256,
      "portMappings" : [
          {
              "hostPort": 0,
              "containerPort": 2000,
              "protocol": "udp"
          }
       ]
    }

AWS X-Ray SDK for Python

The X-Ray SDK for Python is a library for Python web applications that provides classes and methods for generating and sending trace data to the X-Ray daemon. Trace data includes information about incoming HTTP requests served by the application, and calls that the application makes to downstream services using the AWS SDK, HTTP clients, or an SQL database connector. You can also create segments manually and add debug information in annotations and metadata.

You can download the SDK with pip.

$ pip install aws-xray-sdk

Encryption

AWS X-Ray always encrypts traces and related data at rest. When you need to audit and disable encryption keys for compliance or internal requirements, you can configure X-Ray to use an AWS Key Management Service (AWS KMS) key to encrypt data.

X-Ray provides an AWS managed key named aws/xray.

Resource Policy for Amazon SNS Tracing by AWS X Ray

{
    Version: "2012-10-17",
    Statement: [
      {
        Sid: "SNSAccess",
        Effect: Allow,
        Principal: {
          Service: "sns.amazonaws.com",
        },
        Action: [
          "xray:PutTraceSegments",
          "xray:GetSamplingRules",
          "xray:GetSamplingTargets"
        ],
        Resource: "*",
        Condition: {
          StringEquals: {
            "aws:SourceAccount": "account-id"
          },
          StringLike: {
            "aws:SourceArn": "arn:partition:sns:region:account-id:topic-name"
          }
        }
      }
    ]
  }

AWS X Ray IAM policy

AWSXRayDaemonWriteAccess – Write permissions for using the X-Ray daemon, AWS CLI, or AWS SDK to upload segment documents and telemetry to the X-Ray API.

    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "xray:PutTraceSegments",
                "xray:PutTelemetryRecords",
                "xray:GetSamplingRules",
                "xray:GetSamplingTargets",
                "xray:GetSamplingStatisticSummaries"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

The following is an example of an identity-based permissions policy for a CreateGroup action

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "xray:CreateGroup"
            ],
            "Resource": [
                "arn:aws:xray:eu-west-1:123456789012:group/local-users/*"
            ]
        }
    ]
}

CloudWatch Monitoring of AWS X Ray

You can use the X-Ray SDK for Java to publish unsampled Amazon CloudWatch metrics from your collected X-Ray segments. These metrics are derived from the segment’s start and end time, and the error, fault, and throttled status flags.

Deploy the CloudWatch agent to collect metrics using Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Container Service (Amazon ECS), or Amazon Elastic Kubernetes Service (Amazon EKS):

Configure the SDK to communicate with the CloudWatch agent. By default, the SDK communicates with the CloudWatch agent on the address 127.0.0.1.

com.amazonaws.xray.metrics.daemonAddress=address:port

Conclusion

In this article you read everything about AWS X Ray.