The course is intended for individuals looking to understand the basics of software engineering as they relate to building large software systems that leverage big data. You will be introduced to software engineering concepts necessary to build and scale large, data intensive, distributed systems. Starting with software engineering best practices and loosely coupled, highly cohesive data microservices, the course takes you through the evolution of a distributed system over time.
An app for verifying email addresses in a registration flow, which is designed to handle very high throughput.
Run docker-compose.
docker-compose up
Run migrations
./gradlew devMigrate testMigrate
Use the Gradle Kotlin plugin to run tests, build, and fetch dependencies. For example, to build run
./gradlew build
Run the notification server.
./gradlew applications:notification-server:run
Luckily, Gradle fuzzy-matches task names, so the command can optionally be shortened to
./gradlew a:n:r
Run the registration server in a separate terminal window.
./gradlew applications:registration-server:run
Run the fake Sendgrid server in another separate terminal window.
./gradlew platform-support:fake-sendgrid:run
Post to http://localhost:8081/request-registration to make a registration request. Include the email address to register in the request body.
{
"email": "[email protected]"
}
Don't forget to add the content type header.
Content-Type: application/json
Check the logs of the fake Sendgrid server for your confirmation code. Once you receive it, post to http://localhost:8081/register to confirm your registration. Include your email address and confirmation code in the request body.
{
"email": "[email protected]",
"confirmationCode": "18675309-1234-5678-90ab-cdef00000000"
}
Don't forget to add the content type header.
Content-Type: application/json
See the requests.http
file for sample requests
The benchmark app runs a simple benchmark test against the running apps.
Stop the fake Sendgrid app, then run the benchmark app with
./gradlew applications:benchmark:run
This will send some traffic to the notification and registration servers, and will print some basic metrics to the console.
Once the benchmark is finished, try running it again giving different values for the REGISTRATION_COUNT
,
REGISTRATION_WORKER_COUNT
, and REQUEST_WORKER_COUNT
environment variables.
After getting comfortable with the environment, try running multiple instances of the notification server and the
registration server.
Make sure to provide a unique PORT
environment variable to each instance of the registration server.
Now that we have our system working with multiple instances, we will implement a consistent hash exchange
to better distribute load between our registration request consumers.
Look for the TODO
s in the codebase to help you get started.