• Skip to primary navigation
  • Skip to main content

antpb

Just another WordPress dev

  • Home
  • Music
  • Dev Blog
  • Bio
  • Donate

Development

Journey to Pure Data in VR using Unity3D

antpb · Dec 2, 2017 · Leave a Comment

I’m on a quest! I want to integrate Pure Data with Unity to make a Virtual Reality game focused on performing music. I’m fairly new to game development. My prior experience with game dev was when I was in my pre teens making Flash games. For this quest, I want to focus on this project being catered to performers….at least for now. How that will look? I’m not sure but I have built a framework to start with. I’m very aware  there are many iterations that are currently in development or have been around for some time, and I admire those projects immensely. This will just be my flavor…

Backstory!

Lets get some back story on how I got here. Music with event based controls over 3D Visuals or better yet, Virtual Reality worlds, has been a dream of mine since I started making music. I was heavily inspired by Summer Wars and dabbled in Blender. I learned to model by giving myself the task of recreating pieces of the Land of OZ from Summer Wars. Sadly I’ve lost the file so I cant do much for it anymore.

Concept Art

After familiarizing myself with 3D modeling I set my sights on developing a concept for this project.This was around 2014 and at this time I created a music video using the cube in this render. I mapped the scaling animations to the peaks in the audio file for the song. Not much going on there but it technically was my first audio controlled world.

Unity!

I began exploring Unity3D and found it to be very capable for sound purposes. I got sound running and instantiated audio reactive cubes across the spectrum.

Learnings!

I continued documenting the project via Instagram as I learned new things. I’ve explored going the iOS/mobile route and found it to be somewhat limiting in user input.

View this post on Instagram

Progress intensifies

A post shared by Anthony Burchell (@antpb) on Jun 4, 2016 at 10:06pm PDT

Controllers!

A year passed and while still learning, progress was pretty much at a stand still. I acquired a Vive and very quickly got my prior proof of concept loading. I then shifted to exploring controller input and how to attach it’s properties to other objects. This allowed for picking up objects. Small step but was very important.

View this post on Instagram

Getting a little better at this unity stuff. Avoiding prefabs as much as possible. Here I'm just testing some of the physics properties and controller actions. Got all my controller states mapped out. Plan to move on to making animation change based on controller state (glow differently when you hold things)

A post shared by Anthony Burchell (@antpb) on Jun 30, 2017 at 10:47pm PDT

Sound manipulating objects!

Using Keijiro’s many frameworks I was able to get sound manipulating a 3D object! I split out a song over three tracks (drums, synth, percussion) and began experimenting with what a live song would look/feel like.

View this post on Instagram

Got live audio routing now. Working on improving the latency. These photos are the progression of this concept since 2014. The first render is where I'm at now. Photo 2-3 is the initial concept from '14 done in blender. The following photos was my progression bringing it into a headset on a budget. It's been so fun. Progress is so much quicker now that I have a production machine/ vr setup and especially since the foundation is now built. . . . #vr #virtualreality #unity3d #indiedev

A post shared by Anthony Burchell (@antpb) on Aug 27, 2017 at 2:15pm PDT

Shaders!

I became obsessed with iridescent shaders in the process of learning this stuff (partially because my girlfriend turned me on to how cool iridescence looked) I found that this type of shader was extremely good for understanding how to script color and lighting.

View this post on Instagram

Update on the vr project. Loving these iridescent shaders . . . #glitch #idm #electronicmusic #3d #unity3d #irridescent #vr #vrdev #virtualreality

A post shared by Anthony Burchell (@antpb) on Sep 27, 2017 at 2:47pm PDT

Pure Data!

Pure Data has been an eye opening experience. I had built very basic sequencer/osc a few years ago and stopped looking into it due to the complexity of the routings. I found recently that an open source framework called Automatonism simplified a lot of my problems with Pd. I've since moved away from this framework but it was an excellent resource for prototyping.

View this post on Instagram

My first pure data patch in automatonism. Full vid on my YouTube channel: theantpb 🙂 . . . #automatonism #puredata #electronicmusic #synthesizer

A post shared by Anthony Burchell (@antpb) on Nov 21, 2017 at 7:37pm PST

Pure Data and Unity!

After experimenting for a day I was hooked. I took a weekend to learn everything I could. I found that there was an amazing C# library called libpd and compiled myself a 64bit version of the library for use in Unity3d. The results? INPUT AND CONTROL OVER SOUND! My most significant progress towards this dream.

View this post on Instagram

Yessssss got my pure data generative patch loading in unity. Now I can make accompanying sequences and parameters changes over my static backing tracks for this VR album idea I've had. You'll be able to manipulate the song by interacting with objects 😀 . . . #gamedev #puredata #unity3d #vrdev #electronicmusic

A post shared by Anthony Burchell (@antpb) on Nov 21, 2017 at 11:54pm PST

View this post on Instagram

Got my generative pure data patch talking with unity3d! First example is the bang triggering kick drum in pure data standalone, next is game loading the patch and sending the bang message on press of space bar. After that I route the same space bar message to the start clock input of the automatonism clock. 😀 . . . . #unity3d #gamedev #indiedev #puredata #electronicmusic #automatonism

A post shared by Anthony Burchell (@antpb) on Nov 23, 2017 at 11:01pm PST

View this post on Instagram

Worked on this over the holiday break 🙂 this is pure data controlled by unity3d. You can trigger a basic synth and send bang events to switch patch routing in PD. . . . . #puredata #gamedev #indiedev #unity3d #electronicmusic #virtualreality #vr #vrdev

A post shared by Anthony Burchell (@antpb) on Nov 27, 2017 at 7:46am PST

Proof of Concept

After going on a code binge I was able to bring to life a very simple proof of concept using the same Pd patch shown above. This gives the basic building blocks for what I need.

View this post on Instagram

Woot! Sliders are now controlling pure data! . . . #puredata #unity3d #unity #indiedev #indiegamedev #vrdev #virtualreality #electronicmusic

A post shared by Anthony Burchell (@antpb) on Nov 28, 2017 at 9:42am PST

View this post on Instagram

The control center! Knobs are working 😀 . . . . #vrdev #unity3d #puredata #indiedev #gamedev #virtualreality

A post shared by Anthony Burchell (@antpb) on Nov 29, 2017 at 7:21am PST

It’s starting to look and feel like something.

After a few years of breaking through the learning curve of C#, Pure Data, and Unity3D, I finally have a working game that feels like something I would be excited to perform in.

I’ll be continuing to update my blog as new things develop on this project.

View this post on Instagram

Let there be sequencers and envelopes! . . . . #puredata #vr #vrmusic #unity3d #automatonism #indiedev #virtualreality #electronicmusic

A post shared by Anthony Burchell (@antpb) on Dec 1, 2017 at 12:13am PST


  • « Go to Previous Page
  • Go to page 1
  • Go to page 2

A website made with WordPress on WP Engine