initialCapacity[]

Fundamentals of Software Architecture for Big Data

The course is intended for individuals looking to understand the basics of software engineering as they relate to building large software systems that leverage big data. You will be introduced to software engineering concepts necessary to build and scale large, data intensive, distributed systems. Starting with software engineering best practices and loosely coupled, highly cohesive data microservices, the course takes you through the evolution of a distributed system over time.

← Back

Download the codebase

Provenance metrics

In this exercise we'll look at introducing service level indicators within the Provenance codebase.

The Exercise

Capture Provenance metrics!

Quick start

After downloading the codebase, you'll notice that the provenance-metrics project includes a few example metrics. Search for article-requests to review the Meter metric.

Then review the Dropwizard metrics Getting Started page to gain a general understanding of the new code. Dropwizard Metrics is a Java framework for developing ops-friendly, high-performance, RESTful web services built by Coda Hale and the engineering crew at Yammer.

Build the provenance-metrics project.

./gradlew clean build

Run the server using the below command.

java -jar applications/provenance-server/build/libs/provenance-server-1.0-SNAPSHOT.jar

Prometheus

We'll be using Prometheus to store our metrics data. Prometheus is an open-source monitoring application built by the engineers at SoundCloud.

Install Prometheus and ensure that Prometheus is configured with our new metrics endpoint.

brew install prometheus

Modify /usr/local/etc/prometheus.yml to match the example below. For homebrew modify /opt/homebrew/etc/prometheus.yml.

  - job_name: 'dropwizard'
    metrics_path: '/metrics'
    scrape_interval: 5s
    scheme: http
    static_configs:
      - targets: [ 'localhost:8881' ]

Restart prometheus.

brew services restart prometheus

Upon success, you should see our Dropwizard endpoint http://localhost:8881/metrics UP on the Prometheus Status Targets page.

Prometheus target

Grafana

Grafana allows you to query, visualize and alert on metrics from a variety of data sources. We'll be using Grafana to display our metrics data stored in Prometheus.

First, install and run Grafana.

brew install grafana
brew services restart grafana

Then use the web application provided by Grafana to set up and configure our Prometheus data source.

Prometheus data source

Add your newly created prometheus data source http://localhost:3000/datasources/new. Use http://localhost:9090 for your grafana data source http url.

Next, create a new dashboard and graph our article_requests_total data.

Use the query below to display requests per second.

irate(article_requests_total[5m])

Grafana query

Curl the provenance articles endpoint to start tracking metrics. User a bash script similar to the example below to drive multiple requests per second.

while [ true ]; do for i in {1..10}; do curl -v -H "Accept: application/json" http://localhost:8881/articles; done; sleep 5; done

Your dashboard should start recording data.

Grafana dashboard

Capture a screenshot of your Prometheus targets page and Grafana dashboard to complete the exercise.

Hope you enjoy the exercise!

Thanks,

The IC Team

© 2022 by Initial Capacity, Inc. All rights reserved.

A workshop by

Initial Capacity