DALEC Developer Experience: Declarative Builds and IDE Integration

Key Takeaways

Because Dallek is built on top of BuildKit, you can use all your standard tools like the Docker CLI, whatever scanning infrastructure that you have. Hello and welcome to another exciting episode of Open at Microsoft. In today's episode, we will take a journey through Dallek. And today we are joined by Brian, who is principal engineer in Dallek team. Welcome, Brian. Hello. So, for those who are new to Dallek, what solution space in Dallek is trying to solve in today's container and cloud native ecosystem? Sure. So, Dallek is a build tool that you can execute with Docker build. It It's kind of a alternative to using a Dockerfile and it works with all your existing tooling. The goal of it is to essentially take control of your supply chain. So, it makes it relatively easy to see what you're bringing into a build and also to restrict the kind of things that you're actually pulling into that build. Oh, very cool. So, how does Dallek differs from the traditional approaches like Dockerfile or BuildKit in terms of the reproducibility or developer experience? Yeah, so Dallek is actually built on top of BuildKit. So, as I mentioned before, you Docker build your your Dallek spec, which is a YAML document, and it just has a special syntax parameter at the top, which if you use Dockerfile, you probably have seen that before, where to set the right version of the Dockerfile format and what have you. So, we hook into that same thing and the way Dockerfile and and Dallek work are very similar. We're taking some instructions you gave us and sending it off to BuildKit to actually execute the build graph. Where we differ is Dallek is very It will Dallek is declarative. Dockerfile is an imperative layered system. So, you say, "Oh, get this and copy this thing in and run this command and do all this kind of stuff." Dallek is very declarative and you say, "Here's what I'm bringing into the build. Here's my dependencies um that are not source based like like Go or something like that. And here's what I'm producing. And because of that, we're able to take advantage of certain things like, "Oh, you told us you have a Go project." We can go ahead and prefetch those build dependencies or sorry, those Go module dependencies. We can set up build caching for you and all the things that that you would have to handle manually in a Dockerfile. The focus is really on system packages versus um just kind of putting together what you want. So, we we rely heavily on the system package manager to bring in dependencies and we also produce packages and then we create containers from those packages. So, it kind of integrates with the whole system package ecosystem. Yeah, really cool. I mean, it kind of shims all the details which are required for the BuildKit and I see how it can be valuable for the end user being more declarative and helping user throughout the journey. Um Where does you think Dallek fit in a broader supply chain security and provenance which comes with it? Do you have a demo to share, La? Yeah, sure. I can go ahead I'll go ahead and start the demo. Um and while it's going, I can talk. So, um because Dallek is built on top of BuildKit, you can use all your standard tools like the Docker CLI, whatever scanning infrastructure that you have. Uh BuildKit produces SBOM provenance and it actually works extremely well with Dallek because as you can see from the source So, actually what's I'll actually explain what's happening here. I've told uh Docker to build this spec file. So, we have a set of sources that we're bringing in, which in this case it's just a Git repository at a specific commit. I've told it this is a Go module. So, the reason why we do this is we actually disable network access during builds. So, um stuff like Go module um downloads for your for your Go dependencies won't actually work down here. So, we tell it this is a Go module. Dallek takes care of fetching those for you. It It caches them. So, you don't have to like fetch it again and again every time. Um and that's kind of what we're seeing here. So, we built uh this is building a Debian package right now or built a Debian package originally. It's building an RPM now all from the same spec file, which as you can see is just fairly basic. Um this because it's just Go, it's relatively easy build. The same package name for both AlmaLinux and for Ubuntu Noble, which are the two things that we're building here. Um but if we needed to have separate things like say what if Go was named different depending on the different distributions that we're targeting, you can certainly fit that into the spec, which is here. Uh so, we define a set of inputs, our our build dependencies, and then we have some build steps. I just basically build this binary. We call it Go MD to Man. And then we tell Dallek we have this artifact at MD to Man {slash} Go MD to Man. And that's how it knows how to fit it into your package that it's building. Uh and then we have, you know, the image stuff. So, if you need to configure an entry point or add environment variables or all the kinds of things that you would do to an image, add labels, that kind of thing. Uh that's that modifies how the container is built. And you can see we're going to build a uh AlmaLinux container there uh from that. So, it's actually taking the package that we built, the RPM that we built, and putting that into AlmaLinux. And it also used AlmaLinux as the build toolchain. So, the package, if you're dynamically linked or something like that, it will work on AlmaLinux. And you can output packages, you can output containers. Um we really aren't focused too much on what you what the actual build outputs are. We want to support what you need and um so, right now that's pretty much packages and containers and minimal containers at that. As you can see, the the container doesn't have shell, it doesn't have a package manager, it just has the the dependencies that we declared, which in this case is just our Go binary cuz it's statically compiled Go binary. Nice. Nice. This is really cool. So, what are the biggest challenges currently exist for Dallek so far for adaptation and for somebody who is new, what will be the great entry point for those engineers? Yeah, so one of the things I would say for for the things that we work on, we end up building a bunch of upstream open source stuff. Like we build Kubernetes, we build all kinds of ecosystem packages and containers, which means we have to go and tease apart those build systems. Some of them, you know, they use Dockerfiles, others are using you know, random scripts and all these kind of things. So, we have to tease apart those and really understand those builds instead of just consuming those those artifacts, just consuming those Dockerfiles and Docker building the Dockerfiles. We have to pull them apart, figure out what they're actually doing, and and make that fit into the Dallek spec format. Now, it's a challenge, but it's also really good thing to have cuz so, when we when we are building our stuff, we know exactly how that build is happening and we're understanding what they did to make their build work and we're making it work for for our needs here. Nice. And to start with, do we have any tools which we can which we can leverage for Dallek spec generation at the moment? >> So, um Yeah, so we do publish a JSON schema which can be used for like LSP style helping helpers for InVim or VS Code. And we also have a VS Code extension that kind of goes above and beyond the LSP stuff. It integrates with the VS Code debugger with help from Docker Build X and you can There's code lenses and all kinds of nice things for for managing your your spec file. Oh, that's pretty cool. That might be a great second episode for this for actually looking into the VS Code and how that makes life easy using Dallek extension. I see you have a documentation on this on the side. Do you want to share um for the folks to also Yeah. I should have done this a little bit differently. So, yeah, we have pretty thorough documentation. If we're missing something, of course, would love to have people let us know, which you can come to our GitHub repo and let us know about that. Um kind of goes through a quick start for, you know, building similar things like we did with Go MD to Man. And it also has some more advanced builds as well. And how to do virtual packages and even like I didn't even mention that we we support building system B system extensions, which are like overlays for your root of S that allow all kinds of stuff. Um it'll kind of go through the whole thing, the specifications and uh all that. This is pretty cool and very nice. This documentation looks really nice as well. So, my final question will be looking ahead, what does success for Dalek looks like in a couple of years? Particularly within the CNCF ecosystem. Yeah, so I think one thing is we are taking advantage of container build systems or traditionally container build systems, but we're not really focused on containers and we want to be able to build whatever kind of artifact you need, which is um right now it's RPMs and DEBs for us and and containers. And we're adding system extensions, um which you can see there. Uh so, we're not really focused on the container thing and we basically want to be able to produce the outputs that you need and and you tell us the inputs and we'll support the outputs that you need and um hopefully make the middle build part kind of as seamless as possible. Now, that's pretty awesome. And for those who wants to raise a feature request and uh other things from the uh open source community, uh what is the right place and how they should reach out? Yeah, so that's going to be uh on GitHub uh project.dalek/dalek. We are open to issues, pull requests. Uh we have a full test suite if you want to write write code um that can work on your machine or anywhere. It works uh should work on Mac, Linux, Windows. Uh which is simple go go test. Uh no special setup. Um and of course we have some other repositories in the org as well for like the VS Code extension and other things. So, uh but the Dalek the main Dalek repo is the place I would definitely send people to. This is amazing. I really appreciate this for sharing this and I hope everybody will will find it very helpful as they start their Dalek journey. Thank you so much Brian for sharing your Dalek knowledge and I really looking forward for the next episode and the progress that I have on this area. Great. Thank you so much.