diff --git a/docs/assets/code/c/src/TransientFederates.lf b/docs/assets/code/c/src/TransientFederates.lf new file mode 100644 index 000000000..2e388214a --- /dev/null +++ b/docs/assets/code/c/src/TransientFederates.lf @@ -0,0 +1,78 @@ +target C + +preamble {= + #include + #include +=} + +/** Persistent upstream federate. Sends 0, 1, 2, ... every 2 seconds. */ +reactor Up(period: time = 2 s) { + output out: int + timer t(0, period) + state count: int = 0 + + reaction(t) -> out {= + lf_set(out, self->count); + lf_print("Up sending %d", self->count); + self->count++; + =} +} + +/** + * Transient federate that forwards inputs from `Up` to `Down`. + * After four inputs it leaves the federation by calling `lf_stop()`. + */ +reactor Middle { + input in: int + output out: int + output join: int + state count: int = 0 + + reaction(startup) -> join {= + tag_t t = lf_tag_start_effective(); + lf_print("Middle joined at effective start tag (" PRINTF_TIME ", %u)", + t.time - lf_time_start(), t.microstep); + lf_set(join, 0); + =} + + reaction(in) -> out {= + self->count++; + lf_print("Middle forwarding %d (count %d)", in->value, self->count); + lf_set(out, in->value); + if (self->count == 4) { + lf_stop(); + } + =} +} + +/** Persistent downstream federate. Continues even while Middle is absent. */ +reactor Down(period: time = 2 s) { + timer t(0, period) + input in: int + input join: int + + reaction(t) {= + lf_print("Down timer at (" PRINTF_TIME ", %u)", + lf_time_logical_elapsed(), lf_tag().microstep); + =} + + reaction(join) {= + lf_print("Down observed Middle join"); + =} + + reaction(in) {= + lf_print("Down received %d from Middle", in->value); + =} +} + +federated reactor { + up = new Up() + down = new Down() + + @transient + mid = new Middle() + + up.out -> mid.in + mid.join -> down.join + mid.out -> down.in +} diff --git a/docs/assets/code/py/src/TransientFederates.lf b/docs/assets/code/py/src/TransientFederates.lf new file mode 100644 index 000000000..56077bd89 --- /dev/null +++ b/docs/assets/code/py/src/TransientFederates.lf @@ -0,0 +1,76 @@ +target Python + +preamble {= + import os + import subprocess + import sys +=} + +# Persistent upstream federate. Sends 0, 1, 2, ... every 2 seconds. +reactor Up(period=2 s) { + output out + timer t(0, period) + state count = 0 + + reaction(t) -> out {= + out.set(self.count) + print("Up sending {}".format(self.count)) + self.count += 1 + =} +} + +# Transient federate that forwards inputs from `Up` to `Down`. +# After four inputs it leaves the federation by calling `lf.stop()`. +reactor Middle { + input inp + output out + output join + state count = 0 + + reaction(startup) -> join {= + t = lf.tag_start_effective() + print("Middle joined at effective start tag ({}, {})".format( + t.time - lf.time.start(), t.microstep)) + join.set(0) + =} + + reaction(inp) -> out {= + self.count += 1 + print("Middle forwarding {} (count {})".format(inp.value, self.count)) + out.set(inp.value) + if self.count == 4: + lf.stop() + =} +} + +# Persistent downstream federate. Continues even while Middle is absent. +reactor Down(period=2 s) { + timer t(0, period) + input inp + input join + + reaction(t) {= + print("Down timer at ({}, {})".format( + lf.time.logical_elapsed(), lf.tag().microstep)) + =} + + reaction(join) {= + print("Down observed Middle join") + =} + + reaction(inp) {= + print("Down received {} from Middle".format(inp.value)) + =} +} + +federated reactor { + up = new Up() + down = new Down() + + @transient + mid = new Middle() + + up.out -> mid.inp + mid.join -> down.join + mid.out -> down.inp +} diff --git a/docs/assets/images/diagrams/TransientFederates.svg b/docs/assets/images/diagrams/TransientFederates.svg new file mode 100644 index 000000000..ad351fac4 --- /dev/null +++ b/docs/assets/images/diagrams/TransientFederates.svg @@ -0,0 +1 @@ +TransientFederatesUp(0, 2 s)outDown(0, 2 s)123injoinMiddle12inoutjoin \ No newline at end of file diff --git a/docs/glossary/glossary.mdx b/docs/glossary/glossary.mdx index 422560dfb..16eb369f7 100644 --- a/docs/glossary/glossary.mdx +++ b/docs/glossary/glossary.mdx @@ -7,6 +7,18 @@ description: Glossary of terms used in the Lingua Franca documentation. Glossary of terms used in the Lingua Franca (LF) documentation. +### Federate +A top-level reactor instance in a [federation](#federation). The compiler generates a separate program for each federate. + +### Federation +A distributed Lingua Franca program, specified with a `federated reactor`. The compiler generates a separate program for each top-level reactor instance (each **federate**) plus, for most targets, an RTI that coordinates startup and shutdown. See [Distributed Execution](../writing-reactors/distributed-execution.mdx). + +### Persistent Federate +A federate that must be present when a federation starts and that remains until the federation ends. Federates are persistent unless they are marked `@transient`. See [Transient Federates](../writing-reactors/transient-federates.mdx). + +### Transient Federate +A federate marked `@transient`. It need not be present when the federation starts, and it may join and leave during execution. Supported for the C and Python targets. See [Transient Federates](../writing-reactors/transient-federates.mdx). + ### LF File A source file with the `.lf` or `.ulf` extension, representing a Lingua Franca (LF) program. The `.ulf` extension is used for [micro-LF](https://micro-lf.org) programs, and the `.lf` extension is used for all other LF programs. diff --git a/docs/reference/target-language-details.mdx b/docs/reference/target-language-details.mdx index fa9833fab..49a25be6b 100644 --- a/docs/reference/target-language-details.mdx +++ b/docs/reference/target-language-details.mdx @@ -1706,8 +1706,9 @@ There are also some useful functions for accessing physical time: - `instant_t lf_time_physical()`: Get the current physical time. - `instant_t lf_time_physical_elapsed()`: Get the physical time elapsed since program start. - `instant_t lf_time_start()`: Get the starting physical and logical time. +- `tag_t lf_tag_start_effective()`: Get the tag at which this federate effectively started. For a [transient federate](../writing-reactors/transient-federates.mdx) that joins a running federation, this may be later than the federation start tag. -The last of these is both a physical and logical time because, at the start of execution, the starting logical time is set equal to the current physical time as measured by a local clock. +The last two of these relate to start time. `lf_time_start()` is both a physical and logical time because, at the start of execution, the starting logical time is set equal to the current physical time as measured by a local clock. A reaction can examine the current logical time (which is constant during the execution of the reaction). For example, consider the [GetTime](https://github.com/lf-lang/lingua-franca/blob/master/test/C/src/GetTime.lf) example: @@ -1951,8 +1952,9 @@ There are also some useful functions for accessing physical time: - `lf.time.physical() -> int`: Get the current physical time. - `lf.time.physical_elapsed() -> int`: Get the physical time elapsed since program start. - `lf.time.start() -> int`: Get the starting physical and logical time. +- `lf.tag_start_effective() -> Tag`: Get the tag at which this federate effectively started. For a [transient federate](../writing-reactors/transient-federates.mdx) that joins a running federation, this may be later than the federation start tag. -The last of these is both a physical and a logical time because, at the start of execution, the starting logical time is set equal to the current physical time as measured by a local clock. +The start time from `lf.time.start()` is both a physical and a logical time because, at the start of execution, the starting logical time is set equal to the current physical time as measured by a local clock. A reaction can examine the current logical time (which is constant during the execution of the reaction). For example, consider the [GetTime.lf](https://github.com/lf-lang/lingua-franca/blob/master/test/Python/src/GetTime.lf) example: @@ -2820,6 +2822,8 @@ For micro-LF documentation, see [micro-lf.org](https://micro-lf.org). A reaction may request that the execution stop after all events with the current timestamp have been processed by calling the built-in method `request_stop()`, which takes no arguments. In a non-federated execution, the actual last tag of the program will be one microstep later than the tag at which `request_stop()` was called. For example, if the current tag is `(2 seconds, 0)`, the last (stop) tag will be `(2 seconds, 1)`. In a federated execution, however, the stop time will likely be larger than the current logical time. All federates are assured of stopping at the same logical time. +To stop only the calling federate, without requesting that the rest of the federation stop, call `lf_stop()`. That is used by [transient federates](../writing-reactors/transient-federates.mdx) to leave a running federation. + > The [timeout](<../writing-reactors/termination.mdx#timeout>) target property will take precedence over this function. For example, if a program has a timeout of `2 seconds` and `request_stop()` is called at the `(2 seconds, 0)` tag, the last tag will still be `(2 seconds, 0>)`. @@ -2827,6 +2831,8 @@ A reaction may request that the execution stop after all events with the current A reaction may request that the execution stop after all events with the current timestamp have been processed by calling the built-in method `lf.request_stop()`, which takes no arguments. In a non-federated execution, the actual last tag of the program will be one microstep later than the tag at which `lf.request_stop()` was called. For example, if the current tag is `(2 seconds, 0)`, the last (stop) tag will be `(2 seconds, 1)`. In a federated execution, however, the stop time will likely be larger than the current logical time. All federates are assured of stopping at the same logical time. +To stop only the calling federate, without requesting that the rest of the federation stop, call `lf.stop()`. That is used by [transient federates](../writing-reactors/transient-federates.mdx) to leave a running federation. + > The [timeout](<../writing-reactors/termination.mdx#timeout>) target property will take precedence over this function. For example, if a program has a timeout of `2 seconds` and `request_stop()` is called at the `(2 seconds, 0)` tag, the last tag will still be `(2 seconds, 0>)`. @@ -2967,10 +2973,11 @@ Reactions in C can use a number of [pre-defined functions](https://www.lf-lang.o - **File Access** - - LF_SOURCE_DIRECTORY: A C string giving the full path to the directory containing the `.lf` file of the program. - - LF_PACKAGE_DIRECTORY: A C string giving the full path to the directory that is the root of the project or package (normally, the directory above the `src` directory). + - LF_SOURCE_DIRECTORY: A string giving the full path to the directory containing the `.lf` file of the program. + - LF_PACKAGE_DIRECTORY: A string giving the full path to the directory that is the root of the project or package (normally, the directory above the `src` directory). - LF_SOURCE_GEN_DIRECTORY: The directory in which generated files and any files in the [files]() target directive are placed. - - LF_FILE_SEPARATOR: A C string giving the file separator for the platform containing the `.lf` file ("/" for Unix-like systems, "\\" for Windows). + - LF_FED_PACKAGE_DIRECTORY: A string giving the full path to the directory that is the root of the federate package (normally, the directory above the `src` directory appended with `fed-gen/name`, where `name` is the name of the federation). + - LF_FILE_SEPARATOR: A string giving the file separator for the platform containing the `.lf` file ("/" for Unix-like systems, "\\" for Windows). These are useful when your application needs to open and read additional files. For example, the following C code can be used to open a file in a subdirectory called `dir` of the directory that contains the `.lf` file: diff --git a/docs/sidebars.ts b/docs/sidebars.ts index c2da9d96f..5206c1098 100644 --- a/docs/sidebars.ts +++ b/docs/sidebars.ts @@ -108,6 +108,10 @@ const sidebars: SidebarsConfig = { "type": "doc", "id": "writing-reactors/distributed-execution" }, + { + "type": "doc", + "id": "writing-reactors/transient-federates" + }, { "type": "doc", "id": "writing-reactors/polyglot" diff --git a/docs/writing-reactors/distributed-execution.mdx b/docs/writing-reactors/distributed-execution.mdx index f2f318689..60af6e943 100644 --- a/docs/writing-reactors/distributed-execution.mdx +++ b/docs/writing-reactors/distributed-execution.mdx @@ -226,6 +226,10 @@ The time value is specified by a positive integer followed by units, one of `ns` When a federate receives the starting time from the RTI, then it will wait until its local physical clock matches or exceeds that starting time. Thus, to the extent that the machines have [synchronized clocks](#clock-synchronization), the federates will all start executing at roughly the same physical time, a physical time close to the starting logical time. + + +By default, the RTI waits for **every** federate to register before choosing that start time. You can instead mark some federates `@transient` so that they need not be present at startup and can join and leave during execution. See [Transient Federates](./transient-federates.mdx). + diff --git a/docs/writing-reactors/termination.mdx b/docs/writing-reactors/termination.mdx index 2c3585aae..5341e8cdf 100644 --- a/docs/writing-reactors/termination.mdx +++ b/docs/writing-reactors/termination.mdx @@ -63,6 +63,10 @@ When the RTI receives a **STOP_REQUEST** message from a federate, it forwards it When a federate receives a **STOP_REQUEST** message, it replies with its current logical time _t_, completes its current tag (if one is progress), and blocks, waiting for a **STOP_GRANTED** message from the RTI. When it gets the reply with payload _s_, if _s_ > _t_, then it sets `timeout` = _s_ and continues executing, using the timeout mechanism (see above) to stop. If _s_ = _t_, then it schedules the shutdown phase to occur one microstep later, as in the unfederated case. + +To stop **only the calling federate** without stopping the federation, call `lf_stop()``lf.stop()`. That is the mechanism [transient federates](./transient-federates.mdx) use to leave while the rest of the federation continues. + + ## External Signal diff --git a/docs/writing-reactors/transient-federates.mdx b/docs/writing-reactors/transient-federates.mdx new file mode 100644 index 000000000..a915351bf --- /dev/null +++ b/docs/writing-reactors/transient-federates.mdx @@ -0,0 +1,195 @@ +--- +title: Transient Federates +description: Federates that can join and leave a running federation. +--- + +import { + LanguageSelector, + NoSelectorTargetCodeBlock, + ShowIf, ShowIfs, ShowOnly, +} from '@site/src/components/LinguaFrancaMultiTargetUtils'; + + + +:::note + +Transient federates are supported for the `C` and `Python` targets only. +They work with both [centralized and decentralized coordination](./distributed-execution.mdx#centralized-coordination), and with both logical and [physical connections](./composing-reactors.mdx#physical-connections). + +::: + +By default, a [federation](./distributed-execution.mdx) waits until every federate has registered with the RTI before it starts, and if a federate leaves, the federation is expected to shut down. That model fits programs that run as a single, fixed ensemble. Many distributed applications need more flexibility: a federate may not be available at startup, may fail and recover, or may be replaced while the rest of the system keeps running. + +A **transient federate** is a federate that is not required to be present when the federation starts and that may join and leave during execution. Federates that are not marked transient are **persistent**: they must join at startup and they remain until the federation ends. At least one federate in a federation must be persistent. + +Typical uses include: + +- Participants that come and go, such as devices that connect to a service only while they are in range. +- Fault recovery, where a federate process is restarted and rejoins without stopping the rest of the federation. +- **Hot swap**, where a new instance of a federate replaces a running one so that software can be upgraded or a failed replica can take over. + +The Lingua Franca IDEs draw a transient federate with a dark green border so that it is easy to distinguish from persistent federates. + +## Marking a Federate Transient + +Apply the `@transient` attribute to a top-level instantiation inside a `federated reactor`: + +```lf +federated reactor { + persistent = new Persistent() + + @transient + arriving = new Arriving() + + persistent.out -> arriving.in +} +``` + +The attribute is allowed only on a federate instantiation (a reactor created directly in the federated main reactor). It is an error to put `@transient` on a nested reactor, on a reactor class, or in a non-federated program. + +## Example: Leaving and Rejoining + +The following program, adapted from the Lingua Franca regression tests, has three federates. `Up` sends integers every two seconds. `Middle` is transient: it forwards those integers to `Down` and, after four inputs, leaves the federation. `Down` has its own timer, so it keeps executing whether `Middle` is present or not. + +import TransientFederatesSVG from "./../assets/images/diagrams/TransientFederates.svg" + + + +import C_TransientFederates from '../assets/code/c/src/TransientFederates.lf'; +import Py_TransientFederates from '../assets/code/py/src/TransientFederates.lf'; + + + +Compile and run this as any other federated program, but use the `--tmux` (or `-x`) command-line option to launch the federation in a tmux session: + +```sh +lfc src/TransientFederates.lf +bin/TransientFederates --tmux +``` + +The generated launch script starts the RTI and every federate, including `mid`. You should see `Middle` join at the federation start tag (or shortly thereafter), forward four values, then leave. The federation has now become disconnected, but `Down`'s timer keeps ticking synchronously with `Up`'s timer. + +You can relaunch `Middle` by just rerunning the program with the same command-line options using `Control-P` in the tmux subwindow for `mid`. + +## Starting a Federation with Transient Federates + +The RTI is told both how many federates exist in total and how many of them are transient. The generated launch script passes these as `-n` (or `--number_of_federates`) and `-nt` (or `--number_of_transient_federates`). For the example above there are three federates, one of which is transient, so the RTI is invoked with `-n 3 -nt 1`. + +The federation starts as soon as every **persistent** federate has registered. Transient federates may register before that, in which case they share the federation start tag, or they may register later. + +The generated `bin/` script launches transient federates along with the persistent ones, which is convenient for testing. You can instead start only the RTI and the persistent federates, and launch each transient later. When you do that, give the transient the same federation ID as the RTI (`-i` / `--id`; see [Federation ID](./distributed-execution.mdx#federation-id)): + +```sh +fed-gen/TransientFederates/bin/federate__mid -i myFederationID +``` + +## Leaving a Federation + + + + +A transient federate leaves in an orderly way by calling `lf_stop()` from a reaction. That stops **only this federate**, at one microstep after the current tag. Unlike [`lf_request_stop()`](./termination.mdx#stop-request), it does not ask the RTI to stop the federation and does not require consensus among federates. Shutdown reactions in the leaving federate run normally at that final tag. + + + + +A transient federate leaves in an orderly way by calling `lf.stop()` from a reaction. That stops **only this federate**, at one microstep after the current tag. Unlike [`lf.request_stop()`](./termination.mdx#stop-request), it does not ask the RTI to stop the federation and does not require consensus among federates. Shutdown reactions in the leaving federate still run at that final tag. + + + + +After the federate exits, the RTI treats it as absent. Persistent federates continue. The same federate ID may join again later, either because you relaunch the same binary or because you start a replacement binary with the same connections (see [Hot Swap](#hot-swap)). + +A disorderly departure — a crash, a killed process, or a dropped network connection — also leaves the federate absent. Centralized coordination can still agree on the tag of the last tagged message the RTI forwarded from that federate. There is no way in general to agree on the tag of the last message on a [physical connection](./composing-reactors.mdx#physical-connections) or with decentralized coordination. + +## Joining or Rejoining + +When a transient federate (re)joins a federation that is already running, the RTI computes an **effective start tag** for it. Persistent federates always start at the federation start tag. A transient that joins later starts at a tag that is at least the federation start tag and is late enough that it does not contradict tag-advance grants or messages that the RTI has already issued (for centralized coordination). + + + + +The joining federate can read that tag with `lf_tag_start_effective()`. Compare `lf_time_start()`, which is the federation start time and does not change when a transient joins late. In the example, `Middle` prints the effective start tag in its `startup` reaction. + + + + +The joining federate can read that tag with `lf.tag_start_effective()`. Compare `lf.time.start()`, which is the federation start time and does not change when a transient joins late. In the example, `Middle` prints the effective start tag in its `startup` reaction. + + + + +Under **centralized** coordination, the effective start tag is the maximum of: + +1. The physical time at which the federate requested to join (as a tag with microstep 0). +2. The federation start tag. +3. One microstep after the last tag this federate completed, if it is rejoining. +4. One microstep after the latest tag-advance grant (TAG or PTAG) already given to any downstream federate. +5. One microstep after the latest message the RTI has seen that was addressed to this federate (including messages dropped while it was absent). + +Pending tag-advance grants to downstream federates at or after that tag are canceled so that the newcomer can participate from its effective start tag onward. + +Under **decentralized** coordination, if the transient has no upstream federates, its effective start tag is the tag it proposes. Otherwise the RTI adds the same startup delay used for the federation start time (currently one second) so that in-flight messages from upstream federates can arrive before the newcomer advances. If network latency, clock error, and execution lag together exceed that delay, a [tardy](./distributed-execution.mdx#maxwait) (safe-to-process) violation is possible; see [Decentralized Coordination](#decentralized-coordination). + +## Timers + +A timer in a transient federate is **not** aligned to the federation start tag. It behaves like a timer in a [modal model](./modal-models.mdx) that has just become active: the first firing is at the federate's effective start tag plus the timer's offset, and later firings follow the period from there. + +If you need alignment with the federation timeline, schedule a logical action from the `startup` reaction using the difference between the current tag and the next aligned time. + +## Absent Federates + +While a transient federate is absent: + +- Messages sent **to** it are dropped. To the sender, it is as if the receiver ignored the message. +- It sends nothing to its downstream federates. Those downstream federates still advance time; they simply see no events on the connections from the absent federate. + +The intervals of absence are well defined: from the federation start tag until the first effective start tag, and then from the tag at which the federate left until the next effective start tag. + +With centralized coordination, the RTI delays tag-advance grants to downstream federates of an absent transient just enough that a joining transient is not forced to wait for a grant that was issued far into the future. If every upstream of a federate is an absent transient, the federate may advance to its next local event. + +With decentralized coordination, federates that are downstream of an absent federate treat inputs from the missing federate as absent at all tags, regardless of their `maxwait` values. When an upstream transient (re)joins, the downstream federates detect the (re)established connection and (re)activate their `maxwait` timers before they advance. + +## Hot Swap + +If a new process connects to the RTI with the same federate ID as a transient that is still running, the RTI performs a **hot swap** instead of rejecting the connection: + +1. The RTI sends a stop request to the old instance. +2. The old instance stops at one microstep after its current tag and sends a RESIGN message. +3. The RTI then accepts the new instance and computes its effective start tag as for any other join. + +Hot swap is allowed only for transient federates, only during the execution phase (not while the federation is still starting), and only one swap at a time. The replacement must present the same neighbor structure (the same connections to other federates) as the original; the RTI rejects a join whose connectivity differs from the first accepted instance of that federate ID. + +To try this with the example, remove the `lf_stop()` / `lf.stop()` call from `Middle` so that the first instance is still running. Then, in another window, relaunch it. The second copy replaces the first. + +Hot swap does not by itself copy state from the old instance to the new one. If the replacement needs the previous state, the application must save and restore it, as in the next section. + +## Preserving State Across Joins + +State variables of a transient federate are those of the process. When the process exits, that state is gone. A standard pattern is to send state to a persistent federate whenever it changes, and to restore it in the `startup` reaction of a later instance. + +The C [TransientStatePersistence](https://github.com/lf-lang/lingua-franca/blob/master/test/C/src/federated/TransientStatePersistence.lf) and Python [TransientStatePersistence](https://github.com/lf-lang/lingua-franca/blob/master/test/Python/src/federated/TransientStatePersistence.lf) tests do this with a persistent `Persistence` federate. `Middle` notifies `Persistence` when it joins. If this is not the first join, `Persistence` replies with the last saved state. Whenever `Middle` updates its state, it sends the new value to `Persistence`. Reaction order matters: the restore reaction must be able to run before the reactions that depend on the restored state. + +## Decentralized Coordination + +With `coordination: decentralized`, federates exchange tagged messages peer-to-peer and the RTI is not on the data path. Transient federates still register with the RTI so that it can compute effective start tags and notify peers when a transient connects or disconnects. + +Federates that are downstream of an absent federate detect that there is no active connection with the upstream transient federate. They treat inputs from the missing federate as absent at all tags, so they need not wait for the `maxwait` timeout to conclude that the input is absent. When an upstream transient (re)joins, the downstream federates detect the (re)established connection and (re)activate their `maxwait` timers before they advance. + +Physical connections (`~>`) to or from a transient federate are always peer-to-peer, even under centralized coordination. An absent destination drops the message; a present destination assigns a tag from the physical receive time as usual. + +## Security + +Transient joins use the same authentication and federation-ID checks as startup. If you enable [`auth`](../reference/security.mdx) or a secure `comm-type`, a transient cannot join without the same credentials as any other federate. The RTI also checks that a rejoining or hot-swapped federate advertises the same connectivity as the original instance. + +A hot-swapped binary is still arbitrary code that the RTI will run as that federate. Treat the ability to launch a replacement the same way you treat the ability to start the federation: restrict who can obtain the federation ID, credentials, and host access. + +## Limitations + +- Supported only in the `C` and `Python` targets. +- `@transient` applies only to top-level instantiations in a `federated reactor`. +- At least one federate must be persistent. The RTI will not start a federation in which every federate is transient. +- Multi-level chains of transients that participate in a cycle are not covered by the current tests; prefer a single transient on a cycle, or keep cyclic peers persistent. +- Independent compilation of a replacement federate is not yet a separate workflow: you generate the replacement from the same (or a compatible) federated program so that IDs, ports, and connections match. + +Note that banks of transient federates are not given special treatment; each instance is a separate federate.