Service and Container Orchestration for .NET with Tye
Tye helps .NET Core developers build applications that are a collection of services and containers. On your local machine Tye runs your .NET services, any containers you need, and a dashboard you can use to inspect this environment. Tye can then deploy your services and containers to your Kubernetes cluster.
Introduction
In December Ryan Nowak started building Opulence, a “white-glove service for .NET Core and Kubernetes”. Later in the month, David Fowler started building Micronetes, “a local orchestrator inspired by Kubernetes that makes developing and testing microservices and distributed applications easier”. A month ago, those projects merged and became Tye.
The README warns that the project is an opensource experiment and so you should
consider every part of the tye experience to be volatile.
Tye is distributed as tye
via dotnet tool install
. It enables a simpler route to building and deploying applications that have dependencies across services and containers. Consider an example where a developer builds two services: a front-end web application and a back-end api service which, are each destined for deployment as containers in Kubernetes. For each service the workflows will often include these for development:
- Develop the service
- Develop the docker image
- Include dependent containers in
docker-compose.yml
- Run some combination of docker-compose and/or the services directly
And these for deployment:
- Build the services
- Build the docker images
- Push images to container registry
- Build/update/apply k8s deployment
You still have to write the service yourself 😁 but, similar functionality to the rest is offered just with tye run
and tye deploy
. The repo has some good docs for a fledgling project and I followed the four tutorials to get an understanding of Tye’s capabilities. The tutorials resulted in a solution running locally and in Azure Kubernetes Service and you can see the source here.
Running locally
Where I would otherwise use docker-compose
to raise dependent containers, tye run
will run the services I’ve added to my .sln
as containers along with any other containers I’ve specified and, display a dashboard that lists containers I’m running with their (real-time!) logs.
If I’ve only got .csproj
I plan running, then it just works. If I need to add containers, like redis
in the image above, I’ll need to tye init
and add the container dependencies to the resulting tye.yml
.
Deploying remotely
Tye uses docker
and kubectl
to deploy, so you’ll need both of those. You’ll also need a cluster and a container registry - I used Azure for both, but you could use docker hub (or any other) for the registry.
Create your cluster and container registry
az group create -n tye
az configure --defaults group=tye
az acr create -n tyecontainerregistry --sku basic
az acr login -n tyecontainerregistry
az aks create -n tyeAksCluster --generate-ssh-keys --attach-acr tyecontainerregistry --node-count 1 --node-vm-size Standard_B2s
az aks get-credentials --name tyeAksCluster
Deploy
kubectl apply -f https://raw.githubusercontent.com/dotnet/tye/master/docs/yaml/redis.yaml
tye deploy --interactive
kubectl port-forward svc/frontend 5000:80
The above starts with nothing and finishes with a Kubernetes cluster hosting my solution. The two commands before the port-forward
are the only tye
-specific commands. The kubectl apply
is required to deploy the redis container, tye
won’t do this on its own. The --interactive
switch on tye deploy
is required to set the secret, which is the redis connection string even though I couldn’t get secrets to work and elaborate on this below
Troubleshooting
For a new project still in active development Tye is both simple and relatively hassle-free, however there were some smaller issues I had to wrangle and I’ll go through these now.
Authentication Exception
During my first deployment I was having issues authorizing with the container registry, and tye deploy
output included this:
Building Docker Image...
Created Docker Image: 'tyeContainerRegistry.azurecr.io/frontend:1.0.0'
Pushing Docker Image...
unauthorized: authentication required
Drats! 'deploy' failed:
'docker push' failed.
This is because docker authentication is case-sensitive on the registry name even though az acr create
is not. I could have found the solution to this issue quicker if I had bothered to read the warning on login:
λ az acr login --name tyeContainerRegistry
Uppercase characters are detected in the registry name. When using its server url in docker commands, to avoid authentication errors, use all lowercase.
Login Succeeded
Solution: use all lower-case when specifying the container registry via tye deploy --interactive
or in tye.yaml
.
Containers stopping during startup
During tye run
I was finding that the redis-cli
container was being stopped and removed before the other services had finished starting.
λ tye run
[15:15:00 INF] Executing application from c:\git\microservice\tye.yaml
...
[15:15:03 INF] backend_ef831592-1 running on process id 20328 bound to https://localhost:55194, http://localhost:55195
[15:15:03 INF] docker logs collection for redis-cli_ebcabba8-9 complete with exit code 0
[15:15:03 INF] Stopping container redis-cli_ebcabba8-9 with ID 932037f4bef8
[15:15:03 INF] Listening for event pipe events for frontend_d3a0253e-b on process id 20216
[15:15:03 INF] Listening for event pipe events for backend_ef831592-1 on process id 20328
[15:15:05 INF] Stopped container redis-cli_ebcabba8-9 with ID 932037f4bef8 exited with 0
[15:15:06 INF] Removed container redis-cli_ebcabba8-9 with ID 932037f4bef8 exited with 0
[15:15:24 INF] Event pipe collection completed for frontend_d3a0253e-b on process id 20216
[15:15:24 INF] frontend_d3a0253e-b process exited with exit code -1
[15:15:24 INF] Event pipe collection completed for backend_ef831592-1 on process id 20328
[15:15:24 INF] backend_ef831592-1 process exited with exit code -1
[15:15:24 INF] docker logs collection for redis_a313f738-6 complete with exit code 0
[15:15:24 INF] Stopping container redis_a313f738-6 with ID 705c44ba28f1
[15:15:24 INF] Stopped container redis_a313f738-6 with ID 705c44ba28f1 exited with 0
[15:15:24 INF] Removed container redis_a313f738-6 with ID 705c44ba28f1 exited with 0
Solution: Install latest tye
from a master
build, see docs.
Can’t find AddTyeSecrets extension method
error: Unable to find package Microsoft.Tye.Extensions.Configuration. No packages exist with this id in source(s): Microsoft Visual Studio Offline Packages, nuget.org
error: Package 'Microsoft.Tye.Extensions.Configuration' is incompatible with 'all' frameworks in project 'c:\git\microservice\backend\backend.csproj'.
It appears this addition was from 5 days ago and is not deployed to feed yet. I’ve raised this issue on the tye repo.
Solution: Build the package yourself:
git clone git@github.com:dotnet/tye`
cd tye
./buid.md -pack
dotnet nuget add source -n tye-local c:\git\tye\artifacts\packages\Debug\
Updating code in Kubernetes
I changed code in my services and deployed to Kubernetes, but after deployment the changes were not evident. This is because the image deployed has the same tag, so the Kubernetes deployment doesn’t pull the updated image. It looks like there’s an attempt to use latest
however it appears that version will never be null because Version
in csproj seems to default to “1.0.0”.
Solution: Add <Version>1.0.1</Version>
to your csproj files and bump them with every deploy.
Secrets are not accessible in Kubernetes
This presents as HttpStatusCode
500 on the frontend
service with:
An error occurred while processing your request.
and in the logs:
fail: Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware[1]
An unhandled exception has occurred while executing the request.
System.Text.Json.JsonException: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
but is actually because the backend
service couldn’t connect to redis:
fail: Microsoft.AspNetCore.Server.Kestrel[13]
Connection id "0HLUQP4KKQCLD", Request id "0HLUQP4KKQCLD:00000001": An unhandled exception was thrown by the application.
StackExchange.Redis.RedisConnectionException: It was not possible to connect to the redis server(s). UnableToConnect on ::6379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 202s ago, v: 2.0.593.37019
Upon investigation I found that the secret is deployed to the container but still not accessible in IConfiguration
. I’ve raised a bug for it on the tye repo.
Solution: Specify CONNECTIONSTRING__REDIS
as an environment variable instead of a secret.
Debugging locally
You can debug locally by specifying tye run --debug *
, and tye
will output the process ids you can use to attach a debugger to. If you’re using vscode you’ll need to open a new window for each attach. If want to open the same folder twice in vscode, you should use Duplicate workspace in new window.
Conclusion
To conclude, Tye is a pretty neat tool that speeds up container wrangling during development and deployment. I’ll definitely look to start using it in projects where I am otherwise using docker-compose
, after which I’ll be positioned to write a post comparing comparing the differences in experience between the two.