Bacalhau jobs currently are not able to dynamically access network resources. They can access network resources up front by declaring them as IPFS or HTTP(S) downloads that are shared amongst all nodes.

There are a few reasons for this:

1. Security. No networking makes it easy to protect against botnets and Bac jobs running DOS attacks.
2. Determinism. Endpoints can return different results to the same request from different nodes. (Atm we don’t actually do anything about this even with our up-front download, but this is at least a solvable problem for us currently.)

There are lots of use cases we currently can’t meet because of this restriction.

Use cases

Note: maintaining these restrictions for general networking (i.e. allowing jobs to open sockets, even with an allow-list) is hard. Where possible, in the following use cases we have considered alternatives to needing to do that.

1. Downloading dependencies as part of a build

A user has come along and tried to build Nix packages on Bacalhau. We (Simon) has also tried to build Rust into WASM using Bacalhau.

We are interested in this use case because:

1. Juan is interested in it too
2. It could let us provide simpler language interfaces on top of a normal network (e.g. bacalhau run rust could translate into a Docker job to compile a binary and then a WASM job to execute it).

Our Nix user told us that there’s little value in trying to run this type of workload on Bacalhau if the dependencies have to be computed up front because doing that is a majority part of running the build. So the user would need access to the package repository for their language in a way their build tools expect (e.g. via an API).

Once challenge is: how do we stop people running pip install bitcoin_miner and sucking up our compute at build time?

Alternatives to general networking

• Do the work to store package repositories on IPFS, keep them up to date, and then provide them as inputs to a build job, maybe with an API proxy… hard!
• Punt this to executor land – allow someone to write build executors that will do building of things. How they make this secure is a problem to be solved by them (with money to be made if they get people using it).
• Allow selective networking for specific package repositories, as a service. Perhaps via a proxy so that the packages they download can be logged and later verified if running concurrently. Get security by placing strict time and resource limits on the container, and by appropriate pricing, so that mining activities can’t be profitable.

2. Rapid communication between jobs