None of the five problems below is exotic. Each one costs days when it goes unnoticed, because a hidden assumption looks true on the bench until the moment it isn’t.

In this article I walk through five lessons from taking an i.MX RT (Cortex-M7) control unit along that path, in the order a product meets them: the mechanism behind each one, and the rule that keeps it from coming back.

1. The board boots on its own, then dies behind the bootloader

The mental model: the boot ROM was quietly doing your setup for you.

A stand-alone application image on the i.MX RT works because the boot ROM reads the image’s boot header and runs its DCD (Device Configuration Data). That is where external SDRAM gets configured. You never have to think about it.

Then you put a bootloader such as MCUboot in front of the application. A signed application image behind a bootloader has no boot header of its own, so nobody runs its DCD. The first stack push or the first .bss clear in SDRAM ends in a bus fault, long before main().

The fix has two parts:

1. Move the SDRAM setup (the DCD) into the bootloader, which the ROM still boots normally.
2. Make the application skip the clock (PLL) re-initialisation, because the bootloader has already done it.

A related trap: in MCUboot’s direct-XIP-with-revert mode, a slot without the image trailer’s magic value is erased at reset. The symptom is the application disappears after a power cycle. Sign the image with the trailer included.

2. Measure the chip, not the documents

The mental model: the flash size is a measurement, not a setting.

The flash size in a linker script, a ticket or a code comment is often copied from an earlier board. Several documents agreeing with each other proves nothing if they all come from the same assumption. Read the part number on the board, and probe the memory with a debugger before you plan a partition map around it.

Then make the answer structural, so it cannot drift again:

1. One header owns the flash partition map.
2. A start-up assert checks it against the linker symbols.
3. A host test parses the linker scripts, so any drift fails in CI rather than on a customer’s device.

The map matters most once you plan field updates. A dual-slot layout needs room for two complete images, next to the bootloader and the data partition, and images grow over a product’s life.

3. Writing the flash you execute from

The mental model: while the flash is busy, it does not exist.

On the i.MX RT the code usually runs straight out of QSPI NOR (execute in place, XIP). A firmware update writes to the same chip. From the moment you send a program or erase command until the flash clears its busy bit (WIP), nothing may be fetched from it: not the interrupt handler, not the vector table, not a helper function the compiler decided not to inline.

The failure is intermittent, which makes it nasty. The update crashes at a different block on every run, and every bench session looks like a different bug.

What fixes it:

1. Run the erase and program routines, and everything they call, from RAM.
2. Keep interrupts masked while the flash is busy. Check the datasheet for how long that is: a page program is short, but a sector erase takes tens of milliseconds, and the worst case is many times the typical value.
3. Invalidate the instruction cache afterwards. The Cortex-M7 fetches speculatively and can have pulled in stale or garbage lines during the write.

4. Treat the field update as a protocol

The mental model: every update request ends in exactly one outcome, and the new version number is the only proof.

A device that reports “update done” has proven nothing. It might reboot into the old image, or not reboot at all. Update logic only gets robust once you treat it as a protocol with explicit rules:

• Every request ends in exactly one outcome: success, a classified failure, or a timeout. Never silence, never two answers.
• After the restart, poll the device for its version with a fixed interval and a hard limit. Only the new version string counts as success.
• Retries must fit the device’s own limits. If the target closes an idle session after a timeout, all your retries together must finish inside it, or the retries themselves break the session.

The bench catches what unit tests cannot: a bootloader step on the target that takes far longer than you assumed, or a frame buffer that does not fit on the task stack you gave it. Measure step times on hardware and check stack high-water marks before you trust the timeouts.

5. Production-ready means surviving the power cut

The mental model: measure the hold-up time before you design the software around it.

When the supply drops, the board runs a little longer on its capacitors. That hold-up time is a hardware property: measure it on a real board, in the worst-case load state, before you design the power-fail path. One wrong part in the hold-up circuit can bring it close to zero.

That number decides the software design. If the worst-case flash erase (lesson 3) is longer than the hold-up time, the emergency path must never start a flash operation. It only finishes or abandons what is already running, and closes the file system.

Then test it properly. Cut power repeatedly while the firmware writes continuously, and check after each cut that the file system is consistent and the last committed version of each file survived.

Summary

1. Own what the ROM did for you. Once a bootloader sits in front, it has to set up memory and clocks.
2. Measure, then make it structural. Flash size, step timings and hold-up time are measurements; a header, an assert and a CI test keep them true.
3. Design for the worst case. The flash write path runs from RAM, updates end in exactly one outcome, and the power-fail path never starts an erase.

💬 Questions, corrections, or your own war story? Leave a comment below.

Most of what you read about SPE is about cables, connectors and PHYs. This post is about the firmware. As soon as devices are chained one behind the other, the software has to answer questions a star network never asks: Which neighbour is the controller? Where am I in the line? What happens to everyone behind me when I reboot?

In this article I go through the firmware lessons from bringing a 100BASE-T1 daisy chain into an industrial control system: one controller, a line of identical nodes, each node an i.MX RT (Cortex-M7) bridging frames between its two Ethernet ports.

The setup

• Physical layer: 100BASE-T1, point-to-point links chained into a line. Power for the nodes travels over the same pair (power over data line, PoDL).
• Nodes: identical hardware and one firmware image for all of them. Each node has two Ethernet MACs, one per PHY.
• Software: an RTOS, with a lower-priority forwarding task and a queue that bridge frames from one port to the other.

1. Only the upstream port decides “controller reachable”

The mental model: a node’s link to the controller is its upstream port. The downstream port is someone else’s problem.

A tempting first version marks the link as up when either PHY has a link: up = upstream || downstream. That looks harmless. But every node except the last one always has a live downstream link, so it never notices that the controller is gone and keeps running.

The rule that works: the link state the protocol sees is the upstream PHY’s current link bit, and nothing else. The downstream bit only controls whether frames are forwarded. Keep the rule as a tiny pure function with host tests, so it cannot quietly regress.

One limit remains, and the diagram shows it: link state is local. A node only sees its own neighbour. Detecting a lost controller everywhere in a long chain needs an end-to-end heartbeat, not link state.

2. Position from the hop count, identity from silicon

The mental model: a node’s identity is fixed, its position is learnt.

With one firmware image for every node, nothing in flash says “you are node 3”. So the two questions are split:

• Identity comes from a unique serial number in silicon, for example from a secure element on each node. The node also derives its MAC address from that serial, setting the “locally administered” bit and clearing the “multicast” bit, before the Ethernet MACs start.
• Position is discovered at run time. The controller sends a discovery request with a hop counter of 0, and every node that forwards it downstream adds 1.

Two details make it robust:

1. Commands go to a position, never to “everyone”. There is no broadcast fallback, because a broadcast reaches the wrong node when the order in the line is not the one that was accepted.
2. Rediscovery waits for the line to settle. Nodes report a new downstream link a limited number of times, and the controller collects answers in a short window after the last link-up instead of reacting to each one.

3. Store the topology only once it is stable

The controller remembers which device sat at which position, so it can spot a swapped, replaced, missing or extra device later. The trap is when to store that map.

If you store the first discovery after power-up, you may store a half-built chain: the nodes further down are still booting. Later, the missing ones come up and get flagged as “extra devices”: a false fault.

The fix is a debounce. Every usable discovery restarts one timer, and the map is written only when the timer fires. A cold start where every node answers the first discovery will not catch the problem. Test with a growing chain, where nodes come up one after another, and check that the map is written exactly once.

Save the map atomically (write a temporary file, sync, then rename), with a CRC so a damaged file is detected.

4. The receive interrupt is one bit: drain the ring

The mental model: “frames arrived” is a flag, not a count.

The i.MX RT Ethernet MAC raises one receive interrupt flag no matter how many frames landed. If the CPU was busy and three frames arrived, the interrupt still fires once. A handler that reads one frame per interrupt falls behind and stays behind, and no error counter moves.

In a daisy chain this hurts more than on a single device: every node behind the slow one also answers late.

The fix: on each interrupt, keep reading until the receive ring is empty, bounded by the ring size. Three companion changes go with it:

• Read and discard bad frames too, so their buffers are freed.
• Size the forwarding queue for a full ring from each port.
• Keep the frame buffer static, off the interrupt stack.

5. Keep slow hardware access off the critical path

The PHYs are managed over MDIO, which is often bit-banged on GPIO pins. Running that inside a high-priority task blocks the CPU for milliseconds at a time.

Move it to a low-priority worker and match the results back by a token. The high-priority task then only queues a request and picks up the answer later, and its worst-case runtime drops from milliseconds to microseconds.

6. Updating every device on the chain

Firmware updates over the chain follow a few simple rules:

1. Update the devices one after another, in list order, and skip devices that already run the target version.
2. Wait for each device to report its new version before moving on. A restarting device can cut off every device behind it.
3. Classify failures: per device (retry, continue, stop) or global (end the batch).
4. Do not show topology faults while the batch runs. Devices dropping out during their own restart is expected.

Staging every device first and restarting them all together might be faster. Until you have measured that the chain survives it, keep the update sequential.

Summary

1. One rule per question: only the upstream port defines “controller reachable”, the hop count defines position, and silicon defines identity.
2. Wait for the line to settle: store the topology, and show faults, only once the chain has stopped changing.
3. The general rule: in a daisy chain, every node’s behaviour affects everyone behind it, so design and test for the line, not for the single device.

💬 Questions, corrections, or your own war story? Leave a comment below.