Oriel — AI & technology worth noticing, every day.

nemotron-halloween

Web page · https://huggingface.co/blog/nvidia/nemotron-halloween

First seen by Oriel on .

AI summary

MonsterHouse is a Halloween projection that uses NVIDIA Nemotron Voice Agent and open models to listen, generate grumpy pirate dialogue, speak it, and animate a garage-door character on one workstation.

  • NVIDIA Nemotron 3 ASR transcribes visitors, Nemotron 3.5 Bolt E4B generates replies, Magpie TTS Zeroshot voices them, and Audio2Face-3D animates the character in Unreal Engine 5.
  • A LoRA trained on thousands of reviewed example conversations shapes the character; application logic tracks visitor state and manages the conversation flow.
  • Streaming text and cached lines reduce response latency: median end-of-speech-to-audio is 1.70 seconds for generated replies and 0.72 seconds for cached responses.
  • Work is split across an NVIDIA RTX PRO 6000 Blackwell Max-Q for language and voice and an NVIDIA RTX PRO 4000 Blackwell for graphics and animation.
  • Costume recognition and visitor tracking use Nemotron 3 Nano Omni, YOLO, and ByteTrack, but real driveway, costume, and night-lighting tests remain to be done; MonsterHouse code is not packaged for release.

#nemotron #voice agents #text-to-speech #real-time animation

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
Hugging Face Bloghttps://huggingface.co/blog/nvidia/nemotron-halloweenhttps://huggingface.co/blog/nvidia/nemotron-halloween Read discussion