Main

The first quantum networks linking several quantum processors as end nodes have recently been realized as physics experiments in laboratories17,18,19,20,21,22,23 and fibre networks24,25,26, opening the possibility of realizing advanced quantum network applications2 such as data consistency in the cloud27, privacy-enhancing proofs of deletion28, exponential savings in communication29 or secure quantum computing in the cloud13,30. Demonstrations either relied on ad hoc software or chose to establish that hardware parameters were, in principle, good enough to support a given quantum network application, although the application itself was not realized6,31,32. Although quantum nodes have been linked at the hardware level17,18,19,20,21,22,23,24,25,26,33, including the design34,35,36,37 and realization38,39 of network stacks to manage entanglement generation, a critical innovation required to make quantum networks useful is lacking: an architecture enabling the execution of quantum applications.

It is a notable challenge to design and implement an architecture that can enable the execution of arbitrary quantum network applications on quantum processor end nodes (Fig. 1) while enabling programming in high-level software that neither depends on the underlying quantum hardware nor requires the programmer to understand the physics of the underlying devices. In the domain of the conventional internet, the possibility of programming arbitrary internet applications in high-level software has led to the realization of radically new communication applications by diverse communities, which had a transformative impact on our society40. What’s more, the advent of programmable hardware and new application areas sparked new fields of computer science research and guided further hardware development (for example, network programming and protocols, distributed systems, internet of things and more). A similar development is underway in quantum computing, in which the availability of high-level programming tools allows a broad participation in developing applications41.

Fig. 1: Application paradigm.
figure 1

A quantum networking application consists of several programs, each running on one of the end nodes59. An end node is a device in a quantum network that executes user applications. A network stack enables entanglement generation between end nodes over a quantum network (Supplementary Fig. 1). The distinct programs at each end node can only interact through: (1) quantum communication (for example, entanglement generation) and (2) classical communication. This allows a programmer to realize security-sensitive applications but prohibits a global orchestration of the quantum execution, such as what we might do in (distributed) quantum computing70 in which a single quantum program is executed on several nodes. Our architecture allows programs to be written in high-level quantum-hardware-independent software and executed on a quantum-hardware-independent system that controls a hardware-dependent system (QDevice; Fig. 2), such as a NV centre node with a diamond chip (photo taken by authors, left images) or a trapped-ion quantum node60 (right images). These platforms constitute physically very different QDevice systems but can both be programmed by our architecture.

Full size image

In realizing the first such system architecture, we overcome all challenges below, including both fundamental challenges that are inherent to quantum network applications at any scale, as well as technological challenges that arise from the current state of the art of quantum network hardware.

Design considerations and challenges

Interactive classical-quantum execution

The execution of quantum network applications requires a continuing interaction between the quantum and classical parts of the execution, including interactions between different programs (Fig. 1). For example, during secure quantum computing in the cloud13,42, the program on the server is waiting for classical messages from a remote client program before continuing the quantum execution at the server. This is in sharp contrast to quantum computing applications, in which a quantum application is a single program that can be executed in one batch, without the need to keep quantum states live while waiting for input from other programs. In quantum computing, only relatively low level and predictable interactions between classical and quantum processing are realized, such as in quantum error correction43 or mid-circuit measurements44. Higher-level classical-quantum interactions in quantum computing45 do not keep qubits live in memory.

We assume that the programs are divided into classical and quantum blocks of instructions (by a programmer or a compiler). Classical blocks consist of local classical operations executed on a conventional classical processor, as well as networked classical operations (that is, sending messages to remote nodes) executed using network devices. Quantum blocks consist of local quantum operations (gates, measurements, classical control logic), as well as networked quantum operations (entanglement generation) executed on quantum hardware. A single quantum block, in essence, corresponds to a program in quantum computing and may contain simple classical control logic, such as for the purpose of mid-circuit measurements44.

Different hardware platforms

Interfacing with different hardware platforms presents technological challenges: at present, a clear line between software and hardware has not been defined, and the low-level control of present-day quantum processor hardware has been built to conduct physics experiments. Early microarchitectures46,47 and operating systems48,49 for quantum computing do not address the execution of quantum network applications. We thus have to define a hardware abstraction layer capable of interfacing with quantum network processors, including present-day setups.

Timescales

It is a fundamental challenge that different parts of such a system operate at vastly different timescales. For nodes separated by hundreds of kilometres, the duration of network operations is in the millisecond (ms) regime and some applications2 need substantial local classical processing (ms). By contrast, the time to execute quantum operations on processing nodes is in the regime of microseconds (μs) and the low-level control (including timing synchronization between neighbouring nodes to generate entanglement50) requires nanosecond (ns) precision.

Memory lifetimes

Present-day quantum network nodes have short coherence times, posing a technological challenge to ensure that operations are executed within the timeframe allowed by the quantum memory.

Scheduling local and network operations

Entanglement is a key resource for quantum network applications2. In contrast to classical networking, entanglement is a form of stateful connection already at the physical layer in which both nodes hold one qubit. Our architecture should allow for the execution of applications at end nodes, and these may be separated by a large quantum network that facilitates entanglement generation between them. This can be achieved by the implementation of a network stack34,35. A technological challenge arises in the integration of such a network stack with the application stack of QNodeOS when using heralded entanglement generation at the physical layer34 (as done in all current demonstrations linking quantum processors25,38). Heralded entanglement generation requires agreement between neighbouring network nodes to trigger entanglement generation in precise time bins34, organized into a network schedule51 that dictates when nodes make entanglement. Such a schedule could be set by a centralized controller51 or by a distributed protocol (see, for example, ref. 34).

It is a technological challenge to manage the interdependencies between the schedule of local operations, and the networked operations, because, in all current processing node implementations22,52, entanglement generation cannot be performed simultaneously with local operations22,53. Although interdependencies may be mitigated in the future54, this implies that we cannot schedule (that is, decide when to execute) the execution of local quantum operations independently of the network schedule.

Multitasking

When executing (quantum) network applications, one node is typically idle while waiting for the other node before it can continue execution. For example, a client program may need to wait for a server to finish processing and send a message. It is hence a fundamental challenge how we can increase the utility of the system by performing multitasking55,56, that is, allowing the concurrent execution of several programs at once to make use of idle times. Consequently, there is a need for managing state and resources for several independent programs, including processes, quantum memory management and entanglement requests.

Architecture

We divide the architecture logically into three main components (Fig. 2; Methods): the classical network processing unit (CNPU) is the logical element responsible for starting the execution of programs and the execution of classical code blocks; the quantum network processing unit (QNPU, realized on classical hardware) is the logical element responsible for governing the execution of the quantum code blocks; the CNPU and the QNPU together form QNodeOS and control the QDevice, which is responsible for executing any quantum operations (gates, measurements, entanglement generation at the physical layer34) on the quantum hardware. On starting the execution, the CNPU creates a process (a well-known concept in classical operating systems57,58) on the CNPU (a CNPU process), registers the program on the QNPU (through the QNPU’s end node application programming interface (API), Supplementary Information section 2.2), which, in turn, creates its own associated QNPU process (including context such as process owner, ID, process state and priority). QNodeOS also defines kernel processes on the QNPU, which are similar to user processes but are created when the system starts (on boot). The CNPU sends quantum blocks to the QNPU in the form of NetQASM subroutines59. Classical control logic in quantum blocks is executed by the QNPU processor. Quantum gates and measurements (from any QNPU process) and entanglement instructions (from the network process) are delegated to the QDevice by submitting physical instructions (Methods), after which the QDevice responds back to the QNPU with the result of the instruction (Supplementary Information).

Fig. 2: QNodeOS architecture.
figure 2

a, QNodeOS consists of a classical network processing unit (CNPU) and a quantum network processing unit (QNPU) (classical system). QNodeOS controls a QDevice (quantum hardware and low-level classical control). b, Schematic of our implementation of QNodeOS on a two-node setup in which both QDevices control a single qubit in a diamond NV centre. The CNPU is implemented on a general-purpose PC and the QNPU on an embedded system, connected by means of Gigabit Ethernet (blue). The QNPU connects to its QDevice by means of a serial peripheral interface (SPI) (pink). The two QNPUs (brown) and the two CNPUs (green) connect to each other by means of Gigabit Ethernet. The setup is based on ref. 38 with two QDevices (including AWGs and microcontroller units (MCUs); QDevices communicating over a classical digital input/output (DIO) interface) and a heralding station composed of a balanced 50:50 beam splitter (whose output ports are connected to superconducting nanowire single-photon detectors (SNSPDs) through optical fibres (red)), a time tagger (TT) and a complex programmable logic device (CPLD) that heralds the entanglement generation between QDevices and sends a classical message to the MCU.

Full size image

To enable different hardware platforms, we introduce a QDriver realizing the hardware abstraction layer for any hardware corresponding to our minimal QDevice system model (Methods). The QDriver is the only hardware-dependent element of the architecture and is responsible for translating quantum operations, expressed in NetQASM59, into platform-dependent (streams of) physical instructions to the underlying QDevice. We realize a QDriver for the trapped-ion system of refs. 60,61 and one for NV centres in diamond based on the system of refs. 7,22,38. We validate the trapped-ion QDriver (Fig. 3) by implementing and verifying a set of single-qubit gate operations (Methods) and the QDriver on the NV system as part of the full-stack system evaluation (see below).

Fig. 3: Trapped-ion QDevice implementation.
figure 3

Schematic of our implementation of QNodeOS on a single-node setup in which the QDevice contains a single trapped-ion qubit. The QNPU QDriver is implemented on a field-programmable gate array (FPGA) that connects to its QDevice through a SPI (Methods). The setup consists of an emulator that translates between SPI messages and TTL signals, experimental control hardware that includes a FPGA and direct digital synthesis (DDS) modules, a trapped-ion qubit60 under ultrahigh vacuum (Fig. 1) and a photomultiplier tube (PMT) that registers atomic fluorescence.

Full size image

To allow for different timescales, we logically divide the architecture into CNPU, QNPU and QDevice, which can thus be realized at different timescale granularities. In our proof-of-concept implementation, we realize the CNPU and QNPU on different devices, reflecting the millisecond timescales of communication between distant nodes (Methods).

Ensuring the necessary interactivity requires architectural as well as implementation choices: as programs may depend on messages from remote nodes, the architecture needs to be able to dynamically handle both classical and quantum blocks, even if not known at runtime. Consequently, it is not possible to preload all quantum blocks of the program into the low-level controller of the QDevice ahead of time, as done in previous physics experiments. Instead, in our system model, the QDevice is capable of executing individual physical instructions similar to a classical CPU. Consequently, the QNPU is continuously ready to receive new NetQASM subroutines from the CNPU and the QDevice can continuously receive and respond to physical instructions from the QNPU (Methods).

In our NV QDevice implementation, we address the challenge of interactivity by interleaving specific user-requested pulse sequences (realizing physical instructions sent from QNodeOS) and dynamical decoupling (DD) sequences (protecting quantum memory from decoherence) in an arbitrary waveform generator (AWG)62. The DD sequences extend qubit coherence times up to Tcoh = 13(2) ms, while arbitrary physical instructions can be handled by triggering the corresponding pulse sequence, without knowing them in advance (Methods).

To integrate local operations with the network schedule, our architecture first introduces a QNPU scheduler that can choose which of the ready processes is assigned to the local processor (Fig. 2; Methods) and QDevice. This allows interleaving the execution of different processes directly on the QNPU without incurring delays on the timescale of the CNPU (ms), addressing the challenge of short coherence times. In our implementation, we choose to schedule QNPU processes using a priority-based non-pre-emptive scheduler63, owing to limited quantum memory lifetimes, which make it undesirable to pre-empt and temporarily store quantum states while halting the execution. Second, we realize a network process as a kernel process, which manages entanglement generation using the network stack34,35 (implemented in ref. 38 without the ability to execute network applications). Although our architecture can work with any way of setting a network schedule, in our implementation we choose to determine the schedule using a time-division multiple access controller51, allowing the schedule to be centrally optimized to mitigate present-day memory decoherence. The network process handles entanglement requests submitted by user processes, coordinates entanglement generation with the rest of the network through the time-division multiple access controller, interacts with the QDevice and eventually returns entangled qubits to user processes. User processes enter the waiting state when they need entanglement and become ready again once entanglement was delivered. The network process has the highest scheduling priority and, consequently, is given precedence over the execution of any local quantum operations. We remark that local operations may still be executed during time bins already occupied by the network schedule, if a running non-pre-emptable user process prevents the network process from running, as we indeed observe in our evaluation.

To increase utility, QNodeOS allows several programs to be run concurrently, using the QNPU scheduler from above to enable multitasking55,56 user processes on the QNPU itself. The QNPU hence needs to keep context for each process, including a virtual quantum memory space (as in classical operating systems64). Similar to classical memory management systems65, a quantum memory management unit (QMMU) on the QNPU manages qubit allocations from processes and translates virtual qubit addresses in NetQASM subroutines to physical addresses in the QDevice. This allows flexibility in translating a virtual qubit address to: (1) a different physical qubit address over time, allowing qubits to be rearranged transparently in the physical memory in the future or (2) a logical qubit address when QNodeOS is executed on top of a processor using quantum error correction43 (Methods). Entanglement generation between different pairs of processes at remote nodes are distinguished by entanglement request sockets, inspired by classical sockets66,67, which are established once a user process requests entanglement from the network process. In our implementation, processes of the same priority are scheduled first-come-first-served65, where the total schedule of the program in our implementation is dependent on both the schedule on the CNPU as well as that on the QNPU (Methods).

Demonstrations

Delegated computation

We first validate our architecture and implementation by the first successful execution of an arbitrary—that is, not preloaded—execution of a quantum network application in high-level software on quantum processors. We implement QNodeOS on a two-node setup of NV centres using one qubit per node (Fig. 1; Methods). We choose to execute an elementary form of delegated quantum computation (DQC)13 from a client to a server, because the client and server programs jointly realize repetitions of a circuit (Fig. 4a) that triggers all parts of our system (Fig. 4c).

Fig. 4: Delegated computation between two NV centre nodes using QNodeOS.
figure 4

a, DQC circuit (effective computation: single-qubit rotation RZ(α); Methods). The DQC application consists of k circuit repetitions (varying measurement bases for tomography on |ψ⟩) realized by two programs: the DQC-client program (client node, repeating the sequence ‘quantum block (C1, orange)–classical block (computing δ)’ k times) and the DQC-server program (server node, repeating ‘quantum block (S1, blue)–classical block (receiving δ)–quantum block (S2, purple)’ k times). Client and server produce entanglement |Φ+⟩ = (|00⟩ + |11⟩)/√2 (S1 and first part of C1). The client performs gates and a measurement, resulting in outcome bit mc (rest of C1). Client computes δ from mc and DQC parameters α ∈ [0, 2π) and θ ∈ [0, 2π) and sends δ. Meanwhile, the server keeps its qubit coherent. On receiving δ, the server applies gates depending on δ, resulting in single-qubit state |ψ⟩ (S2) depending only on α and θ. b, Experimental results executing DQC for six different sets of (α, θ) parameters (k = 1,200, that is, 7,200 executions of circuit Fig. 4a). Fidelity to |ψ⟩ estimated using single-qubit tomography (1,200 measurement results per data point) and corrected for known tomography errors (SSRO, blue), post-selected for charge-resonance (CR) check validation (purple) and post-selected for latencies (orange) (Methods). c, Sequence diagram including the interaction CNPU–QNPU–QDevice for one execution of the DQC circuit (repeated k = 1,200 times in each experiment) (time flows to the right; not to scale). CNPUs prepare NetQASM subroutines (C1, S1, S2) and send them to their respective QNPUs. CNPUs perform classical computation (message δ). QNPUs execute subroutines, sending physical instructions to their QDevices. Entanglement is generated by QDevices performing a batch of attempts (triggered by a ‘create EPR’ physical instuction), resulting in the heralding of a two-qubit entangled state rotated to |Φ+⟩ by the server. d, Processing times and latencies while server qubit is live (time frame red line in c, averaged over all 7,200 circuit executions except executions with latency spikes; see Methods), including CNPU–QNPU communication latencies, CNPU processing on both nodes and client–server communication latency (CC) (average total of about 4.8(±0.8) ms; error bars (standard deviation) for the sum of individual segments (per segment: Supplementary Information section 4.6)).

Full size image

We first verify that the quantum result (fidelity) was found to be above the classical bound68 >2/3, which verifies that QNodeOS can successfully handle interactive applications consisting of entanglement generation, millisecond-scale memory lifetimes and classical message-passing. The non-perfect fidelity (Fig. 4b) comes mainly from two sources: a noisy entangled state with fidelity 0.72(2) (quantum hardware limitation) and decoherence in the server qubit (depending on Tcoh) as a result of waiting for several milliseconds (classical software latencies; Fig. 4d).

We proceed to characterize latencies. As expected, we find that the duration the server qubit must remain alive is dominated (>50%) by processing in the CNPU, which could be improved by caching the preparation of S2 and implementing the CNPU and the QNPU on one board (see ‘Outlook’ section). We observe that CNPU processing time varies greatly (standard deviation 30%; Supplementary Information section 4.6) owing to limited scheduling control over CNPU processes (Methods). Using an a priori estimate of what delays lead to too low a quality of execution (that is, delays that are too long for the server qubit to be stored with sufficiently high quality), we discard application iterations in which the CNPU latencies spiked by more than 8.95 ms. This leads to the discarding of 2% of iterations in post-processing (Methods).

Demonstration of multitasking

We also validate the multitasking capability of QNodeOS by the first concurrent execution of two quantum applications on a quantum network: the DQC application and a single-node local gate tomography (LGT) application on the client (Fig. 5a). The two programs for the client are started in the CNPU at the same time (two CNPU processes, subject to the CNPU scheduler), which means that the QNPU continuously receives subroutines for both programs from the CNPU (two QNPU processes and corresponding subroutines, subject to the QNPU scheduler). This leads to a multitasking challenge directly on the QNPU to schedule the different subroutines received (Fig. 5b). Because the client has only one qubit, the multitasking of DQC and LGT never results in both programs having a quantum state alive on the client; therefore, multitasking should not affect the fidelity of LGT. We observe interleaved execution of DQC quantum blocks and LGT quantum blocks on the client node (Fig. 5b). The LGT application produces a quantum result (fidelity; Fig. 5c) equal to that in the scenario in which we run LGT on its own (not interleaved by DQC circuit executions), as expected.

Fig. 5: Multitasking experiment on two NV centres with QNodeOS.
figure 5

a, LGT circuit. A single NetQASM subroutine (L1) executes six times for bases B ∈ {±X, ±Y, ±Z}: initialize to |0⟩, rotate around axis D ∈ {X, Y} by angle ϕ, measure in B. The LGT application consists of a single LGT program, submitting L1 to the QNPU (fixed D and ϕ) k times successively. b, Sequence diagram illustrating concurrent execution (multitasking) of DQC (Fig. 4) and LGT on the client: two DQC circuit repetitions (Fig. 4a; two subroutines on the client (orange), four on the server (blue and purple)) and three LGT circuit repetitions (three subroutines, green). The client QNPU receives subroutines for the DQC program and the LGT program, which the QNPU scheduler can interleave: while the server executes S2 (purple), the client can not yet execute the next S1 (orange), as it involves joint entanglement generation. In idle time, the client can execute LGT subroutines (number can vary). c, Results of multitasking LGT (client) and DQC (on both server and client): for each input pair (D, ϕ) ∈ {(X, 0), (X, π), (Y, π/2), (Y, −π/2), (X, −π/2), (X, π/2)} (six cardinal states {±X, ±Y, ±Z}): simultaneously (1) a single LGT program was initiated on the client (k = 1,000); (2) a single DQC-client program was initiated on the client (k = 200 successive subroutines); and (3) a single DQC-server program was initiated on the server (k = 200, that is, 400 successive subroutines), resulting in a total of 6,000 LGT subroutine executions and 36,000 LGT measurement results, yielding fidelity estimates for the LGT quantum state before measurement. The results are the same as running LGT on its own (no multitasking; Supplementary Information section 5.2). d, Scaling number of programs on the client. For N ∈ {1, 2, 3, 4, 5}, we initiate simultaneously: (1) N LGT programs (each using k = 100) on the client; (2) N DQC-client programs on the client (each k = 60); and (3) N DQC-server programs on the server (each k = 60). This results in 2N programs simultaneously active on the client, each continuously submitting subroutines from the CNPU to the QNPU. Each experiment was repeated but with multitasking disabled. The plot shows the utilization factor of the QDevice (fraction of time spent executing instructions), corrected for variable entanglement generation duration (Methods), with (blue) and without (orange) multitasking, showing that multitasking can increase device utilization. Error bars are standard deviation.

Full size image

We further test multitasking by scaling up the number of programs executed concurrently on the client node, up to five DQC and five LGT programs running at the same time on the client. The interleaved execution of subroutines of different programs increases device utilization (fraction of time spent on executing physical instructions) on the client QDevice compared with the same scenario but with multitasking disabled (Fig. 5d). As expected, we observe that LGT subroutines were scheduled to be executed between DQC subroutines, resulting in lower client QDevice idle time. When multitasking one DQC and one LGT program, we observe one or two subroutines between DQC iterations in most cases (LGT subroutine duration approximately 2.4 ms; Supplementary Information section 5.3). We observe cases in which both server and client QDevice remain idle, which could be improved in part by smarter CNPU-QNPU scheduling algorithms: (1) both the client and the server wait until the start of the next network schedule time bin (time bin length 10 ms); (2) the client QNPU finishes a subroutine for user process P but must wait until the CNPU sends the next subroutine for P (up to 150 ms for one DQC and one LGT program, but less (up to only 8 ms) when more applications are running, as there are more CNPU processes independently submitting subroutines); (3) the client is ready to perform entanglement generation for DQC, but the next time bin starts only at some future time t, preventing activation of the network process. The scheduler activates a user process that runs a LGT circuit, which completes at some time >t, delaying the start of the DQC network process, even though the server node was ready at t.

Outlook

We designed and implemented the first architecture allowing high-level programming and execution of quantum network applications. Our architecture does not depend on the distance or connectivity between the end nodes, as long as the network stack enables the use of a quantum network to generate entanglement between them. To deploy our system onto nodes separated by several kilometres, one possible improvement to the implementation of our architecture would be to realize the CNPU and the QNPU on two devices on a single system board, ideally with mutual access to a shared memory to avoid millisecond delays in their communication. Such a merger would also allow the definition of a joint classical-quantum executable and processes, opening further doors to reduce latencies by a better scheduling control. Our architecture could also be used to distribute a quantum computing program over several quantum processors by submitting jobs as NetQASM subroutines to the QNPU of each node.

Our work provides a framework for a new domain of computer science research into programming quantum network applications on quantum processors, including new, real-time69 scheduling algorithms for classical-quantum processes, compile methods for quantum network applications59 or new programming language concepts including entanglement to make software development even easier, thus advancing the vision to make quantum network technology broadly available.

Methods

QDevice model

The QDevice includes a physical quantum device that can initialize and store quantum bits (qubits)—which are individually identified by a physical address—apply quantum gates, measure qubits and create entanglement with QDevices on other nodes (either entangle-and-measure or entangle-and-keep34), either another end node or an intermediary node in the network. We remark that the ability for two end node QDevices that are not immediate neighbours in the quantum network (but that are separated by other network nodes) to generate entanglement between them relies on the architecture implementing a network layer protocol as part of a network stack34. Qubits thereby refers to any possible realization of qubits, including logical qubits realized by error correction. The QDevice exposes the following interface to QNodeOS (Supplementary Information section 2.6): number of qubits available and the supported physical instructions that QNodeOS may send. Physical instructions include qubit initialization, single-qubit and two-qubit gates, measurement, entanglement creation and a ‘none’ for do nothing. Each instruction has a corresponding response (including entanglement success or failure or a measurement outcome) that the QDevice sends back to QNodeOS.

QNodeOS and the QDevice interact by passing messages back and forth on clock ticks at a fixed rate (100 kHz in our NV implementation, 50 kHz in the trapped-ion implementation). During each tick, at the same time: (1) QNodeOS sends a physical instruction to the QDevice and (2) the QDevice can send a response (for a previous instruction). On receiving an instruction, the QDevice performs the appropriate (sequence of) operations (for example, a particular pulse sequence in the AWG). An instruction may take several ticks to complete, for which the QDevice returns the response (success, fail, outcome) during the first clock tick following completion. The QDevice handles an entanglement instruction by performing (a batch of) entanglement generation attempts38 (synchronized by the QDevice with the QDevice of the neighbouring node).

QNodeOS architecture

QNodeOS consists of two layers: the CNPU and the QNPU (Fig. 2a; main text and Supplementary Information). Processes on the QNPU are managed by the process manager and executed by the local processor. Executing a user process means executing NetQASM59 subroutines (quantum blocks) of that process, which involves running classical instructions (including flow control logic) on the local processor of the QNPU, sending entanglement requests to the network stack and handling local quantum operations by sending physical instructions to the QDriver (Fig. 2a). Executing the network process means asking the network stack which request (if any) to handle and sending the appropriate (entanglement generation) instructions to the QDevice.

A QNPU process can be in the following states (see Supplementary Fig. 5 for state diagram): idle, ready, running and waiting. A QNPU process is running when the QNPU processor is assigned to it. The network process becomes ready when a network schedule time bin starts; it becomes waiting when it finishes executing and waits for the next time bin; it is never idle. A user process is ready when there is at least one NetQASM subroutine pending to be executed; it is idle otherwise; it goes into the waiting state when it requests entanglement from the network stack (using NetQASM entanglement instructions59) and is made ready again when the requested entangled qubit(s) is(are) delivered.

The QNPU scheduler oversees all processes (user and network) on the QNPU and chooses which ready process is assigned to the QNPU processor. CNPU processes can run concurrently and their execution (order) is handled by the CNPU scheduler. The QNPU scheduler operates independently and only acts on QNPU processes. CNPU processes can only communicate with their corresponding QNPU processes. Because several programs can run concurrently on QNodeOS, the QNPU may have numerous user processes that have subroutines waiting to be executed at the same time. Hence this requires scheduling on the QNPU.

Processes allocate qubits through the QMMU, which manages virtual qubit address spaces for each process and translates virtual addresses to physical addresses in the QDevice. The QMMU can also transfer ownership of qubits between processes, for example, from the network process (having just created an entangled qubit) to a user process that requested this entanglement.

The network stack uses entanglement request sockets (opened by user programs through the QNPU API once execution starts) to represent quantum connections with programs on other nodes. The entanglement management unit maintains all entanglement request sockets and makes sure that entangled qubits are moved to the correct process.

NV QDevice implementation

The two-node network used in this work includes the nodes ‘Bob’ (server) and ‘Charlie’ (client) (separated by 3 m) described in refs. 7,22,38. For the QDevice, we replicated the setup used in ref. 38, which mainly consists of: an ADwin-Pro II (ref. 71) acting as the main orchestrator of the setup; a series of subordinate devices responsible for qubit control, including laser pulse generators, optical readout circuits and an AWG (Zurich Instruments HDAWG62). The quantum physical device, based on NV centres, counts one qubit for each node. The two QDevices share a common 1-MHz clock for high-level communication and their AWGs are synchronized at the sub-nanosecond level for entanglement attempts.

We address the challenge of limited memory lifetimes by using DD. While waiting for further physical instructions to be issued, DD sequences are used to preserve the coherence of the electron spin qubit72. DD sequences for NV centres can prolong the coherence time (Tcoh) up to hundreds of milliseconds (ref. 7) or even seconds73. In our specific case, we measured Tcoh = 13(2) ms for the server node, corresponding to about 1,300 DD pulses. The discrepancy to the state of the art for similar setups is because of several factors. To achieve such long Tcoh, a thorough investigation of the nuclear spin environment is necessary to avoid unwanted interactions during long DD sequences, resulting in an even more accurate choice of interpulse delay. Other noise sources include unwanted laser fields, the quality of microwave pulses and electrical noise along the microwave line.

A specific challenge arises at the intersection of extending memory lifetimes using DD and the need for interactivity: to realize individual physical instructions, many waveforms are uploaded to the AWG, in which the QDevice decodes instructions sent by QNodeOS into specific preloaded pulse sequences. This results in a waveform table containing 170 entries. The efficiency of the waveforms is limited by the waveform granularity of the AWG that corresponds to steps that are multiples of 6.66 ns, having a direct impact on the Tcoh. We are able to partially overcome this limitation by using the methods described in ref. 74. Namely, each preloaded waveform, corresponding to one single instruction, has to be uploaded 16 times to be executed with sample precision. To not fill up the waveform memory of the device, we apply the methods in ref. 74 only to the DD pulses that are played while the QDevice waits for an instruction from the QNPU, whereas the instructed waveforms (gate/operation + first block of the XY8 DD sequence) are padded according to the granularity, if necessary.

The list of physical instructions supported by our NV QDevice is given in Supplementary Information section 3.1.

NV QNPU implementation

The QNPUs for both nodes are implemented in C++ on top of FreeRTOS75, a real-time operating system for microcontrollers. The stack runs on a dedicated MicroZed76, an off-the-shelf platform based on the Zynq 7000 SoC, which hosts two ARM Cortex-A9 processing cores, of which only one is used, clocked at 667 MHz. The QNPU was implemented on top of FreeRTOS to avoid reimplementing standard operating system primitives such as threads and network communication. FreeRTOS provides basic operating system abstractions such as tasks, intertask message-passing and the TCP/IP stack. The FreeRTOS kernel—like any other standard operating system—cannot however directly manage the quantum resources (qubits, entanglement requests and entangled pairs) and hence its task scheduler cannot take decisions based on such resources. The QNPU scheduler adds these capabilities (Supplementary Information section 2.5).

The QNPU connects to peer QNPU devices by means of TCP/IP over a Gigabit Ethernet interface (IEEE 802.3 over full-duplex Cat 5e). The communication goes through two network switches (Netgear JGS524PE, one per node). The two QNPUs are time-synchronized through their respective QDevices (granularity 10 μs), as these already are synchronized at the microsecond level (common 1-Mhz clock).

The QNPU device interfaces with the QDevice’s ADwin-Pro II through a 12.5-MHz serial peripheral interface (SPI) interface, used to exchange 4-byte control messages at a rate of 100 kHz.

NV CNPU implementation

The CNPUs for both nodes are a Python runtime executing on a general-purpose desktop machine (four Intel 3.20-GHz cores, 32 GB RAM, Ubuntu 18.04). The choice of using a high-level system was made as the communication between distant nodes would ultimately be in the millisecond timescale and this allows for ease of programming the application. The CNPU machine connects to the QNPU device through TCP over a Gigabit Ethernet interface (IEEE 802.3 over full-duplex Cat 8, average ping round-trip time of 0.1 ms), through the same single network switch as mentioned above (one per node), and sends application registration requests and NetQASM subroutines over this interface (10 to 1,000 bytes, depending on the length of the subroutine). CNPUs communicate with each other through the same two network switches.

Scheduler implementations

We use a single Linux process (Python) for executing programs on the CNPU. CNPU ‘processes’ are realized as threads created within this single Python process. Python was chosen because the NetQASM SDK is implemented in Python. When running several programs concurrently, a pool of such threads is used. Scheduling of the Python process and its threads is handled by the Linux operating system. Each thread establishes a TCP connection with the QNPU to use the QNPU API (including sending subroutines and receiving their results) and executes the classical blocks for its corresponding program.

Both the CNPU and the QNPU maintain processes for running programs. The CNPU scheduler (standard Linux scheduler, see above) schedules CNPU processes, which indirectly controls in which order subroutines from different programs arrive at the QNPU. The QNPU scheduler handles subroutines of the same process priority on a first-come-first-served basis, leading however to executions of QNPU processes not in the order submitted by the CNPU (Supplementary Information section 5.3).

Using only the CNPU scheduler is not sufficient because: (1) we want to avoid millisecond delays needed to communicate scheduling instructions across the CNPU and the QNPU; (2) user processes need to be scheduled in conjunction with the network process (meeting the challenge of scheduling both local and network operations), which is only running on the QNPU; and (3) QNPU user processes need to be scheduled with respect to each other (for example, a user process is waiting after having requested entanglement, allowing another user process to be run, as observed in the multitasking demonstration).

Sockets and the network schedule

In an entanglement request socket, one node is a ‘creator’ and the other is a ‘receiver’. As long as an entanglement request socket is open between the nodes, an entanglement request from only the creator suffices for the network stack to handle it in the next corresponding time bin, that is, the ‘receiver’ can comply with entanglement generation even if no request has (yet) been made to its network stack.

Trapped-ion implementation

The experimental system used for the trapped-ion implementation is discussed in refs. 60,61 and is described in detail in ref. 77. The implementation itself is described in ref. 16. We confine a single 40Ca+ ion in a linear Paul trap; the trap is based on a 300-µm-thick diamond wafer on which gold electrodes have been sputtered. The ion trap is integrated with an optical microcavity composed of two fibre-based mirrors, but the microcavity is not used here. The physical-layer control infrastructure consists of C++ software, Python scripts, a pulse sequencer that translates Python commands to a hardware description language for a field-programmable gate array (FPGA) and hardware that includes the FPGA, input triggers, direct digital synthesis (DDS) modules and output logic.

QNodeOS provides physical instructions through a development FPGA board (Texas Instruments, LAUNCHXL2-RM57L (ref. 78)) that uses a SPI. We programmed another board (Cypress, CY8CKIT-143 (ref. 79)) that translates SPI messages into TTL signals compatible with the input triggers of our experimental hardware.

The implementation consisted of sequences composed of seven physical instructions: initialization, RX(π), RY(π), RX(π/2), RY(π/2), RY(−π/2) and measurement. First, we confirmed that message exchange occurred at the rate of 50 kHz as designed. Next, we confirmed that we could trigger the physical-layer hardware. Finally, we implemented seven different sequences. Each sequence was repeated 104 times, which allowed us to acquire sufficient statistics to confirm that our QDriver results are consistent with operation in the absence of the higher layers of QNodeOS.

Metrics

Both classical and quantum metrics are relevant in the performance evaluation: the quantum performance of our test programs is measured by the fidelity F(ρ, |τ⟩) of an experimentally obtained quantum state ρ to a target state |τ⟩, in which F(ρ, |τ⟩) = ⟨τ|ρ|τ⟩, estimated by quantum tomography80. Classical performance metrics include device utilization Tutil = 1 − Tidle/Ttotal, in which Tidle is the total time that the QDevice is not executing any physical instruction and Ttotal is the duration of the whole experiment, excluding time spent on entanglement attempts (see below).

Experiment procedure NV demonstration

Applications are written in Python using the NetQASM SDK59 (code in Supplementary Information), with a compiler targeting the NV flavour59, as it includes quantum instructions that can be easily mapped to the physical instructions supported by the NV QDevice. The client and server nodes independently start execution of their programs by invoking a Python script on their own CNPU, which then spawns the threads for each program. During application execution, the CNPUs have background processes running, including QDevice monitoring software.

A fixed network schedule is installed in the two QNPUs with consecutive time bins (all assigned to the client–server node pair) with a length of 10 ms (chosen to be equal to 1,000 communication cycles between QNodeOS and QDevice as in ref. 38) to assess the performance without introducing a dependence on a changing network schedule. During execution, the CNPUs and QNPUs record events including their time stamps. After execution, corrections are applied to the results (see below) and event traces are used to compute latencies.

DQC

Our demonstration of DQC (Fig. 4) implements the effective single-qubit computation \(| \psi \rangle =H\,\circ \,{R}_{Z}(\alpha )\,\circ \,|+\rangle \) on the server, as a simple form of blind quantum computing that hides the rotation angle α from the server when executed with randomly chosen θ and not performing tomography. The remote entanglement protocol used is the single-photon protocol81,82,83 (Supplementary Information section 3.1).

Filtering

Results, with no post-selection, are presented including known errors that occur during the tomography single-shot readout (SSRO) process (Fig. 4b, blue) (details on the correction in the Supplementary Information of ref. 22). We also report the post-selected results in which data are filtered on the basis of the outcome of the charge-resonance check84 after one application iteration (Fig. 4b, purple). This filter enables the elimination of false events, specifically when the emitter of one of the two nodes is not in the right charge state (ionization) or the optical resonances are not correctly addressed by the laser fields after the execution of one iteration of DQC.

Further filtering (Fig. 4b, latency filtered) is performed on those iterations that showed latency not compatible with the combination of Tcoh of the server and the average entangled state fidelity. For this filter, a simulation (using a depolarizing model, based on the measured value Tcoh; Supplementary Information section 4.4) was used to estimate the single-qubit fidelity (given the entanglement fidelity measured above) as a function of the duration the server qubit stays live in memory in a single execution of the DQC circuit (Fig. 4a). This gives a conservative upper bound of the duration as 8.95 ms, to obtain a fidelity of at least 0.667. All measurement results corresponding to circuit executions exceeding 8.95 ms duration were discarded (146 out of 7,200 data points).

Other main sources of infidelity not considered in this analysis of the outcome include, for instance, the non-zero probability of double excitation for the NV centre83. During entanglement generation, the NV centre can be re-excited, leading to the emission of two photons that lower the heralded entanglement fidelity. The error can be corrected by discarding those events that registered, in the entanglement time window, a photon at the heralding station (resonant zero-phonon line photon) and another one locally at the node (off-resonant phonon sideband photon).

Finally, the dataset presented in Fig. 4b (not shown chronologically) was taken in ‘one shot’ to prove the robustness of the physical layer, therefore no calibration of relevant experimental parameters was performed in between, leading to possible degradation of the overall performance of the NV-based setup.

The single-qubit fidelity is calculated with the same methods as in ref. 8, measuring in the state |i⟩ and in its orthogonal state |−i⟩, provided that we expect the outcome |i⟩, whereas the two-qubit state fidelity is computed taking into account only the same positive-basis correlators (XX, YY, ZZ).

Multitasking: delegated computation and LGT

In the first multitasking evaluation, we concurrently execute two programs on the client: a DQC-client program (interacting with a DQC-server program on the server) and a LGT program (on the client only) (Fig. 5). The client CNPU runtime executes the threads, executing the two different programs concurrently. The client QNPU has two active user processes, each continuously receiving new subroutines from the CNPU, which are scheduled with respect to each other and the network process.

Estimates of the fidelity (Fig. 5b) include the same corrections as in the Supplementary Information of ref. 22. To assess the quantum performance of the LGT application, we used a mocked entanglement generation process on the QDevices (executing entanglement actions without entanglement) to simplify the test: weak-coherent pulses on resonance with the NV transitions, which follow the usual optical path, are used to trigger the complex programmable logic device in the entanglement heralding time window. This results in comparable application behaviour for DQC (comparable rates and latencies; Supplementary Information section 5.1) with respect to multitasking on QNodeOS.

Multitasking: QDevice utilization when scaling number of programs

We scale the number of programs being multitasked (Fig. 5d). We observe how the client QNPU scheduler chooses the execution order of the subroutines submitted by the CNPU. DQC subroutines each have an entanglement instruction, causing the corresponding user process to go into the waiting state when executed (waiting for entanglement from the network process). The QNPU scheduler schedules another process ((56%, 81%, 99%) for (N = 1, N = 2, N > 2)) of the times that a DQC process is put into the waiting state (demonstrating that the QNPU schedules independently from the order in which the CNPU submits subroutines). The number of consecutive LGT subroutines (of any LGT process; LGT block execution time approximately 2.4 ms) that is executed between DQC subroutines is 0.83 for N = 1, increasing for each higher N until 1.65 for N = 5, showing that indeed idle times during DQC are partially filled by LGT blocks (Supplementary Information section 5.3).

Device utilization (see ‘Metrics’ section) quantifies only the utilization factor between entanglement generation time windows to fairly compare the multitasking and non-multitasking scenarios. In both scenarios, the same entanglement generation processes are performed, which—hence—have the same probabilistic durations in both cases. To avoid inaccurate results owing to this probabilistic nature, we exclude the entanglement generation time windows in both cases.