NihraNatural Interaction · Human–Robot AuthoritySee every robot move before it happens.
Direct robots by speaking and pointing. Nihra shows the AI's plan as a hologram in the real workspace first. Nothing moves until a person approves it.
Interactive simulation · not footage · pick a scenario or let them play
AI can read intent. Trust is the missing piece.
A plausible plan is not automatically a safe or correct one.
Programming industrial robots takes experts and setup time. Language models now let people simply say what they want, but the robot still has to get it right.
Nihra pairs AI reasoning with a spatial interface, so a person can see, judge and stop what the robot intends to do, right where it happens.
From MirrorLabs to Nihra. The research started in 2020 with MirrorLabs, mixed-reality digital twins of robot cells (IEEE AIVR 2020). It led to MIRAI, a research prototype built around one arm and one headset, published under that name at APMS 2026. As the approval layer, the server and the interfaces outgrew the prototype, it became a platform with a name of its own: NIHRA, for Natural Interaction, Human–Robot Authority.
Anywhere a person should keep the final say
Wherever a robot acts on AI-generated plans, the same approval layer applies.
Pick, pack and sort
Direct picks by voice, or approve a whole sorting plan at once.
Working next to people
Stops that wait for an explicit resume, handovers that release only on a confirmed grip.
Rehearse in the twin
Dry-run a plan on a digital twin of the cell before it reaches the hardware.
HRI studies and teaching
A working platform for XR-based human–robot interaction research.
Nihra is a research platform, not a certified safety system. Machine safety remains the responsibility of the integrator.
The robot runs exactly what you approved
Nihra is a set of loops between a person and the robot: you see what it understands, it asks when it is unsure, you watch the exact motion before you decide, and the robot reports back. Every task is recorded.
- Instruction to motion
- Feedback to the operator
- Approval: the only way to motion
- Written to the task log
- Operator
Speak, type, point
Say or type the task, click objects or spots in the 3D or camera view and refer to them as “this” or “here”. XR clients on HoloLens 2 and Android connect to the same server.
- Console · Nihra core
See what Nihra sees
The console shows a live 3D scene built from the depth camera and the robot's own pose: objects, surfaces, obstacles and the arm. What is not in that scene, Nihra cannot plan with.
- Nihra core
Understand, or ask back
A language model turns the task into steps that refer to objects in the scene, never coordinates. If something is unclear, it asks instead of guessing.
- Nihra core · Edge
Ground and plan
Every position comes from the scene. MoveIt then plans the exact trajectory on the robot's own computer.
- Operator
Replay, approve or reject
The exact motion plays as a 3D ghost arm; replay it or scrub through it. Approve unlocks only after the whole preview, for that plan only. Reject with a correction and you get a new plan.
- Edge
Execute and report back
The robot runs only the approved trajectory while its joint states and status stream back live. If anything changed, it refuses and tells you why.
Traceable by design
Every task is recorded with timestamps: the instruction and what was pointed at, the scene description the model worked from, the plan it produced, the operator's approval or rejection with the correction, and the outcome or the reason for a refusal. That is the kind of record EU rules increasingly expect for AI-driven machines, in the AI Act and the new Machinery Regulation: human oversight you can show afterwards instead of reconstructing it. Whether a machine complies depends on the complete system and its integrator.
A working proof of concept, end to end
Under active development. This is how it got here, in real recordings.
-
01 · Prototype
First prototype in Unity, called MIRAI
AI-planned actions become a holographic preview instead of running straight away. Shown in the Unity editor. This prototype, MIRAI, is the one described in the APMS paper; it grew into Nihra.
-
02 · Deployment
Running on HoloLens 2
The full system runs end to end on a HoloLens 2, and now also on Android and as a simplified build in the browser. The operator inspects the plan, then approves or rejects it with a correction.
-
03 · Orchestration
Server and supervision interface
One interface for tasks, scene context, AI provider, execution state and approvals, from spoken instruction to robot program.
Published at APMS 2026
The architecture is published in the APMS 2026 proceedings (IFIP, Springer).
"MIRAI: Introducing a Human-in-Command Architecture for Multimodal AI-driven Human-Robot-Interaction"
doi.org/10.1007/978-3-032-38618-2_2 ↗Related research
Beyond the single arm
Simulator-ready Exploring
Isaac Sim, MuJoCo, Gazebo or Unity over MQTT, to rehearse before real hardware.
Autonomous and humanoid robots Exploring
The same approval layer for agentic robots and humanoids.
What people usually ask
Ask me directly about Nihra. You'll usually hear back within two working days.
Ask a questionDo I need a HoloLens 2 to use Nihra?
No. Nihra runs on a HoloLens 2, on Android devices, and as a simplified build in the browser.
Which robots and AI models does it work with?
Robots connect through ROS, and the AI provider is configurable on the server.
What stops the AI from doing something unsafe?
A person approves every plan, the operator can intervene at any time, and local safety analysis runs next to the robot.
Is Nihra a finished product?
No, it is a working proof of concept. Pilots are scoped around a concrete task and robot, or a simulation of one.
Want a human-in-command pilot on your robot?
Describe the task. You'll get a straight answer on whether Nihra fits.