top of page
AI Music Visualize Plugin (Sold & Under Protection)
Side project sold to Beijing JinChang ShiJie Technology Co., Ltd.

Overview

The AI Music Visualize Plugin for Unreal Engine 5 is a cutting-edge tool that seamlessly bridges audio and visuals. It automatically analyzes scene elements suited for music-driven effects—such as lighting changes, dynamic motion, and environmental triggers—by leveraging real-time audio feature extraction, music emotion recognition, and WebSocket-based external integrations. This allows for synchronized visuals in live performances, electronic music festivals, and immersive installations.

Role: Lead Audio Systems Engineer & Technical Project Coordinator                

Time: 2024.5|7 months                                       Team: 10 members (4 time zones)    

Platform: External Software Call                       Tools: Unreal Engine 5, ClickUp

Key Contributions

1. Advanced Audio Analysis & System Development

  • Independent Research & Innovation: Designed and implemented an advanced audio analysis system from scratch, surpassing existing online resources and industry standards.

  • Feature Extraction & Processing:

    • Developed a multi-dimensional audio feature extraction pipeline for analyzing frequency spectrum, amplitude, dominant pitch, and dynamic intensity in real time.

    • Built the entire analysis system using Unreal Engine Blueprints and Python, ensuring precise synchronization with scene elements.

  • Music-Driven Scene Interaction:

    • Designed custom algorithms to process audio signals and dynamically trigger Unreal Engine 5 scene changes, such as lighting intensity shifts, visual effects activations, and object animations.

2. Project Management & Team Leadership

  • Multi-Time Zone Team Coordination:

    • Led a global 10-person team across 4 time zones, ensuring smooth development workflows and timely deliveries.

    • Established a structured management system in ClickUp implementing Sprint Planning, Sprint Backlogs, and milestone tracking.

  • Client Requirements & Technical Documentation:

    • Conducted and finalized requirement analysis, ensuring technical feasibility and alignment with client expectations.

    • Led the team in writing system design documents and user manuals for client handoff.

  • Operational Rules & Process Management:

    • Implemented a structured development framework with:

      • Deadline enforcement & performance-based policies.

      • Clear documentation for requirement changes & client communications.

      • Defined benefit distribution and penalties to maintain accountability.

Impact & Outcome

  • Successfully sold and delivered a high-performance AI-driven music visualization plugin.

  • Developed a unique, research-based audio analysis system unavailable in existing tools.

  • Created a scalable & modular framework, enabling smooth future expansions and integrations.

  • Enabled real-time, music-driven scene synchronization, setting a new benchmark in audio-reactive visuals.

bottom of page