Evidence date · 2024-04-12 · Research note 3 of 3
1 2 3
See current Track status →
Since the previous research note · Partly supported
Track continuity
Since the previous research note: What it takes for a satellite to understand its own detailed images
Previous expectation: Satellites could deliver useful onboard scene classifications once suitable labels and workable hardware made the selected models dependable.
What this study adds: Loris supplies a concrete launched imaging platform, a cloud-labeling proof of concept and reported early thumbnail operations. Its proposed ground-assisted labeling loop addresses adaptation, but the supplied evidence does not establish routine hyperspectral interpretation or measured gains from orbital fine-tuning.
What changed in this study
A small satellite can collect far more image data than it can regularly send to Earth. That mismatch turns space-based artificial intelligence into an engineering problem before it becomes a question of model quality. A camera may see useful detail, but the spacecraft still needs to store, process and transmit the result through a tightly limited communications link.
Loris, an experimental imaging and onboard AI payload on the SpIRIT satellite, was designed around that constraint. The payload combines visible-light and long-wave infrared cameras with a compact computer, image compression and an approach for updating image labels from the ground. SpIRIT launched in December 2023. The April 12, 2024 arXiv preprint discussed here describes the payload’s design and early in-orbit operations, including the use of very small image previews to check image acquisition.
Those activities represent different levels of demonstration. The payload had been launched, its sensors had undergone initial checkouts, and preliminary images had been downloaded. The paper also describes a cloud-labeling proof of concept and a proposed loop for refining the onboard model. Early image transfer shows that the imaging chain was producing usable data for operational checks; it does not by itself show that onboard AI had achieved a validated level of scene understanding.
A camera system built around a small computer
The imaging array installed for the mission contains six Sony IMX219 RGB cameras and three FLIR Lepton 3.5 long-wave infrared sensors. The cameras are placed on multiple sides of the spacecraft, giving the payload more than one viewing direction and more than one type of image measurement.
The Jetson Nano computer at the center of Loris has fewer native camera interfaces than the complete payload requires. A camera-channel multiplexer expands the available connections and can interface with up to 16 sensors. That capacity is larger than the nine sensors installed for this mission. It is useful to distinguish the two numbers: the multiplexer describes the interface design’s potential, while the six visible-light and three infrared cameras describe the flight configuration reported in the paper.
The multiplexer is controlled by an MSP430 microcontroller. It connects the sensors to the Jetson Nano through image-data paths that include MIPI-CSI and SPI channels. This arrangement gives the payload a way to coordinate several cameras while keeping the main computer focused on image handling and AI experiments. The system is therefore a deliberately integrated platform rather than a single camera attached to a general-purpose satellite computer.
That integration also creates a physical problem. The Jetson Nano consumes an estimated 5 watts, and heat behaves differently in orbit than it does in a room or an air-cooled electronics enclosure. In vacuum, convection is not available as a normal route for carrying heat away. Loris therefore relies on conduction through the payload structure and radiation from the spacecraft’s heat-rejection surfaces.
The computer’s CPU, GPU and power-management chip are conductively coupled to an aluminium carrier frame. A compliant space-rated thermal filler helps maintain contact between the board and the frame. The frame serves two roles at once: it acts as a thermal sink that reduces sharp temperature swings as the spacecraft moves between sunlight and eclipse, and it provides a conductive path toward the spacecraft’s heat-rejection system. Thermal simulations were used to examine the computer’s temperature range over an orbit.
The result is not a promise that a processor can simply be placed in space and expected to behave like one on Earth. Computing performance depends on a chain that includes mounting, heat transfer, power management and the changing orbital environment. For an onboard AI payload, thermal design is part of the computing architecture.
Why a tiny preview matters
The first useful image operation described in the paper is deliberately modest. Loris used JPEG-XL image compression with progressive coding, allowing an image to be represented initially by a small preview and then refined as more data becomes available. In early in-orbit operations, thumbnails of about 2 kilobytes were used to evaluate image acquisition, and tens of thumbnails could be downloaded per day.
A 2 kB thumbnail is not a substitute for a full-resolution scientific image. Its value is that it lets operators inspect whether image acquisition is working and decide what deserves more attention before spending more of the available downlink on larger files. The ground team can receive a compact first look rather than waiting for every pixel at the highest available detail.
Progressive coding also changes the order of decisions. Instead of treating every captured image as equally expensive to transmit, the spacecraft-ground system can begin with a lightweight representation. That is particularly useful when an image archive grows faster than the communications link can carry it. The reported thumbnails demonstrate an image-transfer and inspection function; they are not an AI inference benchmark, a measurement of full-resolution throughput or a quantified estimate of mission-wide communications savings.
This is where Loris’s compression work connects to onboard AI without being identical to it. Compression helps decide how much image information can move through the link at a given cost. AI could help decide which images deserve that cost. The early thumbnail result concerns the first task. The paper’s cloud-labeling experiment addresses the second, but the two functions should not be treated as one completed capability.
Compute, memory and storage are separate limits
The Jetson Nano gives Loris a 128-core Maxwell GPU, allowing the payload to explore more parallel processing than a primarily CPU-based system. It is still a small computer by terrestrial AI standards. The reported hardware includes 4 GB of RAM and 16 GB of embedded storage.
Those numbers describe different constraints. RAM is the working space used while software and models are running. Embedded storage holds the operating system, applications, model files and captured data when they are not actively being processed. The paper reports that operating-system and housekeeping requirements leave approximately 2 GB of storage available for operational use.
That remaining space must serve a demanding mixture of tasks. Image collections can be large even before they are transmitted, while an AI model also needs room to load and operate. Larger models with billions of parameters are outside the practical range suggested by this platform’s memory and storage limits. The payload therefore has to balance model size, image retention and the processing needed to produce useful outputs.
The hardware does not make onboard interpretation impossible. It makes the design question more concrete. A model must be small enough to run, the software must leave room for satellite operations, and the data pipeline must prevent stored images from crowding out the rest of the system. The most useful onboard algorithm is not necessarily the most capable model in the abstract; it is one that fits the available resources while producing information that helps manage the mission’s data.
Labels have to travel through a longer route
Loris includes a cloud-detection model as a proof of concept. An onboard label indicating whether an image contains clouds could help estimate the image’s usefulness before the full image is sent to the ground. In principle, that would provide a way to prioritize limited transmission capacity.
The harder question is how such a model can improve after launch. A conventional training workflow depends on reliable labels, but direct downloading of every image is precisely what the payload is designed to make less necessary. Loris’s proposed answer is the Ground Truth Factory at the ground station.
For each captured image, the satellite can record metadata such as the time and GPS coordinates. The ground system can correlate those details with observations from other satellites or from ground-based sources and use the correlation to infer an estimated label. Those labels are then processed on the ground and sent back to the satellite to support further model refinement.
This creates a feedback loop rather than a self-contained learning system. The spacecraft supplies image metadata and runs the model; the ground supplies labels estimated from related observations; the updated information can then return to the spacecraft. The usefulness of the loop depends on whether those inferred labels are suitable for the images being used. A timestamp and location can connect observations, but they do not guarantee that every estimated label matches the satellite image correctly.
The supplied material does not establish cloud-detection accuracy, a quantified improvement from orbital fine-tuning or a completed retraining campaign. What it does provide is a concrete design for addressing the labeling bottleneck. That is an important engineering step because adapting an onboard model requires more than a processor and a camera. It also requires a credible way to learn from the data the spacecraft is collecting.
What onboard image selection could change for Earth observation
For Earth observation, the practical promise is a more useful selection of images reaching mission teams from a constrained spacecraft. Cloud-related labels could help prioritize which scenes deserve larger transfers, while compact previews could support an early decision about whether a captured image warrants further downlink effort.
That benefit remains conditional. It depends on dependable labels, useful model adaptation, sufficient storage and sustained operation of the imaging and computing payload. The early thumbnail operations show that small previews could support image-acquisition checks in orbit. They do not yet establish routine autonomous selection of scientifically valuable scenes, long-term operational adoption or a ready-made Earth-observation service.
If the labeling loop and onboard processing prove dependable in continued experiments, a satellite could spend more of its limited communications capacity on images that are more likely to matter to people using the observations. Loris’s contribution, as described from the April 2024 perspective, is a launched system that brings the camera interfaces, thermal path, constrained computer, progressive image transfer and ground-assisted adaptation problem into one flight experiment.
Key Sources
- Primary source: Mitigating Challenges of the Space Environment for Onboard Artificial Intelligence: Design Overview of the Imaging Payload on SpIRIT
arXiv · Preprint · 2024-04-12