Challenges in developing CEL Playground with WebAssembly

Introduction

In recent blog posts, we've discussed ValidatingAdmissionPolicy, the new Kubernetes API for policy enforcement, and have introduced CEL Playground, a web environment we developed to quickly test Common Expression Language.

You may access the previous articles for more information:

This article explores the motivations and challenges behind development of the CEL Playground with WebAssembly.

Motivation

There were two main motivating factors which led to development of the CEL Playground. The first was the need for an interactive environment to test and experiment with CEL, and the second is the increasing popularity of CEL, especially after it was adopted and became a part of the Kubernetes ecosystem.

Need

When we started adopting CEL for the Marvin checks, and also because it was already used within Kubernetes in the ValidatingAdmissionPolicy API, we could see there was a lack of an environment where CEL expressions could be easily tested and experimented with. Similar purpose languages, such as Rego and CUE, already had web playgrounds for this purpose.

A playground simplifies testing and experimentation with a language, as it doesn't require any prior installations or configurations. Users can directly write and run expressions directly from their browsers, get inspiration from preexisting examples, and also share their experiments.

Popularity

As mentioned earlier, another significant motivation for the creation of CEL Playground was CEL's increasing popularity, especially after it was adopted by Kubernetes, one of the most influential and widely used technology platforms, containing a vast ecosystem.

CEL is used within Kubernetes APIs to declare validation rules, policy rules, and other constraints or conditions. One of the goals of Kubernetes is to provide this functionality as a library so that other projects may run the same CEL expressions as the API Server (the Kubernetes component responsible for this).

New projects, such as Marvin and Kyverno, can already be observed having adopted CEL.

How does the CEL Playground work?

One of the main concerns during development of the CEL Playground was to avoid server costs, and that's where WebAssembly (WASM) has played a crucial role, enabling execution of Go code directly within the browser. As a result, CEL expressions are executed using cel-go.

Whenever the Run button is clicked, the JavaScript function run() is called. This function calls another JavaScript function eval(exp, data) providing the CEL expression and input data as arguments.

Notice that you can inspect and read the source code of the run() function. However, you cannot see the eval(exp, data) function because it is part of the WASM binary, which is not directly visible in the browser.

Upon loading of the CEL Playground, a WASM binary file is loaded and executed, with the help of the JavaScript support file from the Go SDK: $(go env GOROOT)/misc/wasm/wasm_exec.js.

In its essence, it's a Go program that has been running since the WASM binary was compiled with Go.

Go program defines the JavaScript function eval(exp, data) and keeps running. This way, the function remains available to be triggered when the Run button is clicked (via the run() function shown above).

func main() {
    evalFunc := js.FuncOf(evalWrapper)
    js.Global().Set("eval", evalFunc)
    defer evalFunc.Release()
    <-make(chan bool)
}

The evalWrapper function, as the name suggests, is a wrapper that allows running a CEL expression within the context of a JavaScript/WASM environment. For more details, check the GitHub repository.

Finally, the command below is used to compile a Go program into WebAssembly.

GOOS=js GOARCH=wasm go build -ldflags="-s -w" -o web/assets/main.wasm cmd/wasm/main.go

WASM binary size

According to the Go documentation, currently, large WASM files are generated by Go, with the smallest possible size being around ~2MB. This size can increase dramatically when importing libraries, with sizes of 10MB+ being common.

In the case of the CEL Playground, the size of the WASM binary has reached a massive 70MB.

One of the recommended ways to reduce the size of the WASM file is to use TinyGo for compilation.

However, we were not successful when trying to use TinyGo with two different versions.

The first attempt was with version v0.27.0, and the problem was with the use of the reflect package by cel-go:

# github.com/google/cel-go/common/types
../../../../../home/tinygo/go/pkg/mod/github.com/google/[email protected]/common/types/map.go:187:24: undefined: reflect.MakeMapWithSize
../../../../../home/tinygo/go/pkg/mod/github.com/google/[email protected]/common/types/map.go:625:20: undefined: reflect.MakeMapWithSize

Version v0.28.0 improved the reflect support and solved the problem we had had in the previous version.

But this time, we faced a different problem:

# k8s.io/utils/net
../../../../../home/tinygo/go/pkg/mod/k8s.io/[email protected]/net/port.go:116:20: undefined: net.ResolveUDPAddr
../../../../../home/tinygo/go/pkg/mod/k8s.io/[email protected]/net/port.go:120:20: undefined: net.ListenUDP

The indirect dependency k8s.io/utils uses the net package from the standard lib, which is not available in TinyGo. This module is indirectly imported due to some Kubernetes CEL libraries available in the Playground.

Considering that this code is not used, we tried forking, removing unnecessary code, and using replace in go.mod. However, we still faced issues with indirect Kubernetes dependencies:

# k8s.io/klog/v2
../../../../../home/tinygo/go/pkg/mod/k8s.io/klog/[email protected]/klog_file.go:113:7: undefined: os.Symlink

When we commented out the code that imports the Kubernetes CEL libraries, it was possible to compile with TinyGO, resulting in a binary of only 1.6MB.

At this point, we faced a dilemma: whether to keep the Kubernetes CEL libraries available in the playground or to load a (much) smaller binary?

The decision wasn't difficult and we chose the first option, enabling users to test the same expressions on the Playground as those used in Kubernetes APIs.

You can find the TinyGo tests in the GitHub repository's tinygo branch.

If you have any suggestions regarding the challenge of reducing binary size and/or using TinyGo, feel free to open an issue.

Even though using TinyGo was not possible, other actions were taken to reduce binary size.
We were able to achieve a file of 10.8MB by compressing the binary with gzip and using some ldflags in the build command:

GOOS=js GOARCH=wasm go build -ldflags="-s -w" -o web/assets/main.wasm cmd/wasm/main.go
gzip --best -f web/assets/main.wasm

Finally, the front-end had to be adapted to decompress the file before instantiating WASM, which was done using the pako library.

Sharing experiments is a useful and common feature in web playgrounds.

The first version of CEL Playground didn't have this option. But it was in fact the very first suggestion for improvement that came along with the challenges of avoiding server costs.

Many playgrounds save experiments to be shared on the server-side. Since we are aiming to keep the application static, the first idea we tested was to encode information to be shared with base64 and putting it in URL. This worked very well.

We've also added compression with gzip to shorten the URL's length. We have not found clear information about the length limit of URLs in browsers. Therefore, the CEL Playground does not limit it.

Conclusion

The goal of this article was to share the experiences and challenges we've faced in developing CEL Playground with WebAssembly and Go. We also wanted to share a bit about the motivations behind and which led up to this project.

We hope this content will contribute to the community in some way. The project's source code is available on GitHub, and any contributions are greatly appreciated.

Also, make sure to visit our other open-source Kubernetes projects: Zora and Marvin.

Introduction​

Motivation​

Need​

Popularity​

How does the CEL Playground work?​

WASM binary size​

Sharing Experiments​

Conclusion​