In August of 2011 I was presented with an amazing opportunity. Possible Productions reached out to me regarding an upcoming project with Bionic League, Deadmau5′s 2011/2012 tour: Meowingtons Hax. Possible Productions or Possible is an art and design practice which aligns filmmaking, fine art, and computer science to tell amazing stories. They are based in Los Angeles.

Possible was put in charge of designing and creating the visual experience for Deadmau5′s show. I met up with Michael Figge and Alex Afzali from Possible and they described to me their vision for the tour’s visuals, and how they wanted their content to be as tightly synced to the show as possible. 

Here is where it gets interesting, there is usually a set song list for the show, however, Deadmau5 sometimes deviates from the set list and mixes on the fly, which essentially throws off Possible’s pre-rendered visual content. Possible told me they wanted all the visuals to sync perfectly to the music. So there was my challenge as the hacker / artist / designer / creative. The only problem was that they needed something in a week. (5 Days of working time, which is insanely short for production software…).

I don’t think I was truly ready for what was bestowed upon me, but I didn’t care and took on the project anyway. I love electronic music (abstract, experimental, glitch, dance, etc), its what drives me and inspires me.

After Figge and Alex gave me the low down on the challenge, I thought of a couple creative application solutions. After our discussions, I realized that their visuals were key in the show, so my role wasn’t to create new visuals, but to figure out how to affect Possible’s visuals to make them feel in sync with the show. So I decided to build an application that would output generative visuals, and these visuals would act as a mask for the pre-rendered content. In addition to the generative masks, Possible wanted the application to be midi-mappable, so if Deadmau5′s toggles a button or moves a fader up and down it would reflect that in the show’s visuals. Moreover my app had to pixel map the visuals so they displayed perfectly on Deadmau5′s stage setup, which was composed of two stealth LED wedges, an LED back wall, 9 small cubes (with LED panels on visible each side of the cubes) and the main cube (that had large LED panels on each visible side as well). 

I decided to develop the application in openframeworks, seeing as that seemed like the most appropriate for the job. Back in 2011 I was comfortable with OF, but I hadn’t created any of my [now] go to libraries/addons to help me accomplish this feat. So in that week of design and development, I had created the infant/pre-alpha versions of ofxUI, ofxLayers, ofxMidiMapper, ofxBeatDetector, ofxGenerative.

In short, the application, dubbed “Rezanator”, was composed of 5 visuals layers or compositions. Rezanator ran one layer at the time, thus Possible could switch the layers whenever they needed a different look.

Each layer generated a unique looking mask. The first layer, “Black Out”, was used for when the visuals needed to be muted or hidden completely, very useful for breakdowns before the drop. The second layer, “White Out” was a pass through layer (meaning all the pre-rendered visual content for the show would show on stage). With these two layers along, Possible could have midi mapped the layer UI buttons and toggled these on the fly to create a flashing effect on stage. The next layer, “Dynamic Mask”, produced simple squares and rectangles which oscillated in size. This layer had controls that allowed Possible to make the shapes oscillate at different rate and offset the phase of each shape to produce the effect shown in the image below and video (Deadmau5 + Rezanator). The size of the shapes could be modified as well, allowing for various different types of visual masking effects.

The last two layers, “Generative Mask” and “Audio Reactive” were the most interesting tho. For the “Generative” layer I created a generative piece composed particles arranged in concentric circles that would repel the mouse position on screen. Thus when the mouse moved within a certain radius of the particles, the particles would reactive by getting pushed in the z-direction (First image in the post and another directly below). The particles were simulated in real-time, thus Possible had control over their damping, homing force (pulling them back to their original position) and the repulsion force scalar (which determined how far the particles were pushed). These parameters were exposed via interface sliders and buttons and were midi-mappable thus allows for various different aesthetics and motions to be achieved in real-time!

The “Audio Reactive” layer was one of the most interesting to program because of how it mapped the visuals on to the cubes on stage. It was the most rewarding because of the visual effect achieved and its tight synchronization with the music. I wanted to make a square move from one face of the main cube to the next, and I wanted it to transition to the next face when a kick was detected. The end visual result made the square look like it was being pushed by the bass and projection mapped perfectly on to the cube’s surface (Check out the image below which hows the stage’s pixel map and how the square visuals mapped on to it). I wrote a beat detector, ofxBeatDetector (couldn’t have done it without this reference), and used that to trigger a pushing force on the square. The force was proportional to the loudness of the audio, thus the louder the kick/drop the further the square traveled. The square’s damping and force’s magnitude scalar was controllable via the UI thus the movement was tweak-able on the fly. This allowed for various different types of hypnotic / stroboscopic effects.

The last task was to integrate my app into Possible’s setup. I used syphon (via ofxSyphon) to send the visuals out so they could be mixed however Possible wanted to mix them. I also allowed for fullscreen mode thus allowing Possible to capture the output from the screen (i.e. using a Blackmagic or Matrox capture card):

After a week of intensive coding and iterating I was happy with the results. I wrote a beat detector, various interface elements, a view / view controller system for openframeworks, midi mapper, and various visual masks. I learned a ton from this project. It was the first of many projects that drove me to develop ofxUI and various other addons for openframeworks to make my development and workflow easier and faster. Its been about two years and my tools have matured quite a bit by being used and extended on various other projects / platforms.

In the end, Possible went on to produce amazing visuals for Deadmau5′s tour.  I am happy that I got to be part of it, thank you Possible, Bionic League, Deadmau5, and VJ Fader (for connecting me with Possible). Here is a behind the scene’s video from Possible:

In mid 2011 I spent many nights working on iCubist at various coffee shops on the west side of Los Angeles (Venice, Santa Monica, and near UCLA). iCubist, an iOS iPad application commissioned by  the Santa Barbara Museum of Art (SBMA) and the Kimbell Art Museum. The app is an interactive exploration of Analytic Cubism, which is the term used to describe the works of art produced by Pablo Picasso and Georges Braque from 1910-12.

Download Link: https://itunes.apple.com/us/app/icubist/id455593141?mt=8

Eik Kahng, the chief art curator at the SBMA, contacted me and expressed interest in building an interactive experience to augment their Analytic Cubism exhibit. I worked with Eik and the team at SBMA to define and design 5 interactive experiences: An interactive timeline of the key events from 1910-12 (the first image above). A high resolution spectral image viewer (which allowed the user to zoom in and out and mix three spectrally unique different images together via sliders).

An interactive splash screen showcasing the sponsors of the app.

A side by side comparison of paintings (with annotation denoting differences and similarities in the paintings) by Picasso and Braque.

Lastly, an interactive painting decomposition / recomposition experience.

I was responsible for the app’s design, interactions, experiences and programming. This was one of my first iOS apps, and I built it almost entirely with openFrameworks. This project taught me a lot and sharpened my UI/UX/Coding skills. I think this is when I went from newb to pseudo-professional.

Because of the highly interactive nature of the app, it took a couple client reviews and user tests to figure out what was natural from a usability point of view, what worked visually with the aesthetic the museum and I were pushing for and what failed completely and needed to be redesigned. I learned patience, and how long it actually takes to ship an almost bug-free consumer facing app.

Designing the interface for the app was the biggest challenge. At the time I didn’t want to mix a lot of Objective-C and C++ so all the interface elements were written in C++ and openGL, this was way before I wrote ofxUI or any other interface code, so this was my way of learning user interface programming, and it also taught me how to balance my desire for making complex visuals and usable touch interfaces.

Although I enjoyed the experience of making the app, at the end of the project I was frustrated, I had worked almost non-stop for 3 months (which was the time estimated for the project) and had produced an alright app. Its not ground breaking, but it did push forward and augment the museum’s exhibit. I should have blogged about it back then, but I didn’t feel like I learned anything from it. Recently I revisited the app and realized I did learn things, maybe they weren’t all design or programming things, but there was a lot of business and freelancing/contracting knowledge I gained:

Key lessons learned:

Design & UI: keep it simple, show the necessary UI elements when needed, use motion and movement to showcase what elements are interactive and whats not. Have a quick user how to experience at the beginning of the app, make sure you get high resolution assets from your client before you begin building a feature that would use it (e.g. the interactive timeline’s assets are tiny), use Apple’s UIKit (don’t write your own, it will take more time to get right…which I did eventually when I wrote ofxUI, which has been in the works for about a year+). Get the clients to okay the design. If they want a redesign, let them know it won’t be free (design takes time, and implementation takes even more time, so if the UI goes through 3 or 4 redesigns its unfair to assume that comes for free).

Code: don’t rely on openFrameworks or a creative coding framework for everything. Make things object-oriented and well designed from an architecture point of view… so when clients request something a feature, its not insanely hard to integrate or implement, determine whether openGL or Cocoa/Native is the way to go early on (my thoughts on this are, if its going to be highly interactive and have rich graphics, openGL is prob. the way to go, otherwise go with Cocoa, its nice and easy). Code as little as needed, the less you have to maintain, the more time you’ll have to work on other things. When you’ve reached your knowledge limit, don’t be afraid to ask for help or point your clients to someone who is even more experienced than you.

Freelance: Multiply the estimated number of hours for a project by 3, bill hourly, and never go with a flat project fee (you’ll be happier and so will your client). Keep calm and collected, learning how to talk and understand clients is probably one of the most valuable things you can do for your career. Stay focused, so if your moon-lighting (freelancing and working full-time elsewhere) make sure the freelance gigs you take on aren’t insane and won’t suck the life out of you. Have weekly checkins, and show the client the work in person, they’ll get more out of it and so will you (getting feedback is super important for succeeding). Take notes and make to-do lists for freelance work, it will help you get organized and feel productive when you cross things off. Use dropbox, testflight, and github. They cost $$$, but they are worth it, especially when transferring files or getting files, getting feedback on beta version of the app, and maintaing a code based project.

I’ve decided to post this project now because I want to share more with the world in hopes to make it better designed. I am proud of this project, I did the best I could do at that time, given its short time frame. Hope people out there enjoyed this app and it taught people about Analytic Cubism!

After hearing Andrew Bell’s (@drawnline) awesome talk about feeding babies with creative code at Eyeo 2012 and talking to Alex Biem (@tangibleint) I was inspired to port ofxUI (made for Openframeworks) to Cinder. If you aren’t familiar with ofxUI, check it out here. ofxUI contains many widgets and layout methods to help place those widgets so to make user interfaces that are more aesthetically appealing, useable, clean, and minimal.

I’ve seen some beautiful work come out of Cinder, check out Robert Hodgin’s work, and been wanting to know if Cinder makes it more straightforward to use shaders, textures, texture coordinates, FBOs, VBOs, and images to create beautiful visual works. One of the main reasons I ported over ofxUI to Cinder is that it was a chance for me to learn more about how Cinder works, how its designed, and how to use it. The rest of this post will cover ciUI, assuming you know nothing about ofxUI or openFrameworks.

DOWNLOAD/FORK ciUI from here: https://github.com/rezaali/ciUI 

ciUI is a Block for Cinder (version 0.8.4) that easily allows for the creation of user interfaces aka GUIs. ciUI also takes care of variable binding, widget layout, spacing, font loading, color themes, saving and loading settings, and widget callbacks. ciUI can be easily customized (colors, font & widget sizes, padding, layout, etc). ciUI widgets are minimally designed, intuitive to use, easily integrated, flexible and customizable.

ciUI is a GL based GUI and uses Cinder’s drawings calls to render its widgets. It integrates into Cinder projects very easily since it follow’s Cinder’s Block ideology. Just drag and drop the Block’s src folder into your project, add the New Media Fett.ttf font into the resource folder in your project, instantiate a Canvas object to add widgets on, and give it a callback function and/or bind variable to the widgets and your set! The README in the ciUI Block folder includes a step by step tutorial on how to use the library in your projects in under 15 minutes. There are many examples included in the download that show how to add widgets, customize their placement, get values from different types of widgets, set widget values, add callback functions for gui events, themes, iOS, saving and loading settings, and more.

Widgets in ciUI:

• Buttons & Toggles (multi-image & single image buttons, label buttons, toggle matrix, radio toggles, Drop Down List)
• Sliders (multi-image, minimal, bilabel, FPS, circle, rotary, range, vertical, horizontal)
• Number Dials
• 2D pads
• Labels, FPS labels
• Text Input Area
• Waveform & Spectrum Graphs
• Moving Plot/Graph
• Image Display, Image Color Sampler

It has been tested on OSX  and iOS (Cinder 0.8.4). It should work on windows, but I haven’t tested it yet. ciUI is open source under an MIT License, therefore you can use it in commercial and non-commercial projects. If you do end up using it, please let me know I would to show examples of it out there in the wild. If you plan to use it in a commercial project please consider donating to help support this Block and future releases/fixes/features.

My interpretation of the emoticon “OMG” for Samsung’s Noteworthy Project: noteworthyproject.com/

These images were created by an interactive real-time audio reactive visual application, written in Processing.

The algorithm behind the structure and aesthetics of the visuals borrows behaviors from nature, i.e. flocking, magnetism, and ant line formations. The visuals are activated by sound (amplitudes, frequencies), and the particle physics are modifiable in real-time, thus various behaviors can be observed.

The application is capable of rendering each image more than 5 feet in length.

My visuals from a web based glsl shader tool/toy that allows for in browser prototyping of audio-reactive visuals. You can edit glsl code in the browser in real-time! The project was created at Art Hackathon SF by Ryan Alexander, Gabriel Dunne, and Reza Ali.

Check out the project at: https://github.com/notlion/audio-shadertoy

ofxUI is an addon for openFrameworks (version 07) that easily allows for the creation of user interfaces aka GUIs. ofxUI also takes care of widget layout, spacing, font loading, saving and loading settings, and widget callbacks. ofxUI can be easily customized (colors, font & widget sizes, padding, layout, etc).

ofxUI is a GL based GUI and uses openFramework’s drawings calls to render its widgets. It integrates into openFrameworks projects very easily since it was designed specifically for OF. Just drag and drop the addon into your project, copy the GUI folder from an example project’s data folder into your data folder, instantiate a Canvas object to add widgets on! The README in the ofxUI addon folder includes a step by step tutorial on how to use the library in your projects in under 10 minutes. There are many examples included in the download that show how to add widgets, customize their placement, get values from different types of widgets, set widget values, add callback functions for gui events,  saving and loading settings, and more.

DOWNLOAD/FORK ofxUI from here: https://github.com/rezaali/ofxUI

 Widgets in ofxUI:

• Buttons & Toggles (Image buttons, label buttons, toggle matrix, Drop Down Menu)
• Sliders (rotary, range, vertical, horizontal)
• Number Dials
• 2D pads
• Labels, FPS labels
• Text Input Area

It has been tested on OSX  and iOS (OF 07). It should work on linux and windows, since its only dependency is openFrameworks. ofxUI is open source under an MIT License, therefore you can use it in commercial and non-commercial projects. If you do end up using it, please let me know I would to show examples of it out there in the wild. If you plan to use it in a commercial project please consider donating to help support this addon and future releases/fixes/features.

This UI library was inspired & influenced by:

- controlP5 (http://www.sojamo.de/libraries/controlP5/)
- GLV (http://mat.ucsb.edu/glv/)
- SimpleGUI (http://marcinignac.com/blog/simplegui/)
- Apple’s User Interface

I was motived to write my own UI addon because I wanted something like ControlP5 (which is easy to get up and running) for Processing for OF. I use ControlP5 a lot and love its minimal aesthetic. I also love simpleGUI for Cinder, but since I primarily code using OF, I developed ofxUI with the intentions that it needs to be minimally designed, intuitive to use, easily integrated, flexible and customizable.

I designed ofxUI so that if I present my work to a client or put out an App on the App store, it would have a decent aesthetic and be presentable to non-hackers/programmers. I really like the widgets in MaxMsp, so I wanted to have something similar so when I prototype in C++ I could easily go about creating a UI that is pleasing to use and works robustly.

In terms of performance, I haven’t compared it to other GUIs out there, but since its written using OF drawing calls and uses Listeners built in to OF, it run very fast and doesn’t take a lot of CPU…atleast when I tested it when my sketches. On my laptop (Apple Macbook Pro 2009) without vertical sync the all widgets example runs upwards of ~350 fps.

When I first started programming with C++ it was difficult for me to use the other GUIs…their either depended on other libraries, which weren’t included in the download, and it wouldn’t work out of the box, or they were complicated to integrate into my projects. Additionally, I was motived to write my own since I wanted something like ControlP5 (which is easy to get up and running) for Processing for OF. I use ControlP5 a lot and love its minimal aesthetic. I also love simpleGUI for Cinder, but since I primarily code using OF, I developed ofxUI with the intentions that it needs to be minimally designed, intuitive to use, easily integrated, flexible and customizable.

Over the past year I worked on several client projects and got a fulltime job (which I am no longer at), which kept me pretty busy. When I did get some time, I ended up experimenting with typography, color, simulations (particles, springs, and fluid), audio-input and simple rule based systems. These images are the results of half a year or so of coding, tinkering, tweaking, manipulating, and massaging algorithms for generative typography. Read more about these images and how these were created.After briefly experimenting with typography and dynamic systems in Nov. 2010, I started to experiment with color after being inspired by Paul Smith’s vibrant color palette. I believe in minimalism, and for a long time I used only monochromatic color palettes. I still believe in minimalism, but utilize color to make things pop and to give them a playful personality. The image below is of a simple particle-spring simulation melting downward because of the effects of gravity, but staying together because of the springs. When color is applied it starts to resemble a matt oil painting.

The first experiment involved extending my earlier experiments to react to audio. I converted the outline of the type of a phrase (usually, I used “REZA” [because of my graffiti design heritage] or “P5″ or the name of the music artist I was listening to while making the work), into a vector or points then convert them into particles and connected them together by springs. My earlier experiments showed that if you make the rest distances of the springs very small and applied gravity to the particles, then the dynamic system visually mimics paint dripping (when the background is not cleared every frame). See below:

The audio buffer influenced each particle’s velocity. Each particle was assigned a specific sample in the audio buffer. The sample’s value  (which ranged from -1 to 1) was used as a scalar for the particle’s velocity vector. This created a vibration in the composition which visualized the song’s waveform as if it was super imposed on the type’s outline. If background clearing was off, then it started to show an aesthetically pleasing waveform history, which I liked very much. See the image below of the word “BJORK”, its my favorite out of these audio-responsive typography experiments.

The second experiment involved using a fluid field to push the particles around. I used a similar procedure as written above to place the particles in the composition and connect them via springs, then I used a fluid simulation, which I wrote using Jos Stam’s Real Time Fluid Dynamic for Games paper), to influence the particle’s velocities. I interacted with the fluid by moving the mouse around. Since the particles were connected with springs, the type’s outline became a multicolored brush which I used to paint. See this image for a fluid-type-painting:

The third experiment extended the previous two experiments by adding behaviors to the particles, such as going back to their original birth position and physical forces like gravity and repulsion. This resulted in what I call a “fluid melt”, where a composition melted, from this to this to this.

The fourth experiment involved giving the particles a mission and simple rules to guide them. No fluid fields or audio-input influenced their movement in this experiment. The mission was to trace out a path (which was the type’s outline). If the particle got close enough to its target point on the path, then it would target the next point on the path. One rule the particles had to abide by was to avoid other particles while trying to get to its target position. The final visual composition resembled a bold bubble font since the particles were trying to avoid each other and go toward the type’s path simultaneously.

The fifth experiment somewhat inverted the approach of the previous experiments. Strips of particles and springs were placed in 3D space to make a mesh like grid of particles. Then a tracer was used to trace out the outline of the type. The particles repelled the traced proportionally to how close they were to it. I used an exponential decay to give the final visuals the aesthetic seen below:

I would post the code for these experiments, however I believe that after a certain amount of algorithm massaging and code tweaking it starts to encapsulate my style and my design sense, not so much my ability to code. The systems used to create these images are fairly simple, springs, particles, typography and color. Anyone who wants to experiment with these things should check out toxiclibs, although I didn’t use that here, its an awesome library for doing so many things, similar to what I have done here. I used PeasyCam and Geomerative for the 3D camera control and getting the type’s outline.

Side Note: Typically, I like to post projects that are a bit more complete, but I am beginning to realize that my projects are never really done, they just evolve, and evolve, and then months go by and I have no time to work on them anymore or even just upload them to flickr or this blog. From now on I will treat this site as a place to post experiments, some client work, and occasionally tutorials on coding, design, generative art, new media, data visualization, web projects, physical hardware, and so on.

Fields are an important concept to understand in generative design, they are the things that can control compositions and create natural pattern formation. In my work, either particles are interacting with each other and defining their movement or they movement is based on a field in the space where they are put, or both. The images in this post were generated by a real-time particle simulation of 5000 charged particles interacting with magnetic fields. This fundamental theory/law is used to derive their motion is based on the Lorentz Law in Fields and Waves. In physics Lorentz Force defines how a charged particle (point charge) is affected by an electromagnetic field. Lorentz Law states that the force on a changed particle or point charge is equal to the charge of the particle times the electric field which it is placed in, plus its charge times its velocity crossed with the magnetic field present. If there is more than one magnetic field, this equation can be solved for each magnetic field and summed to get the total resultant force. One thing to keep in mind is that these equations yield a vector force (i.e.. 3D), hence these are 3D renderings. These images were generated solely using the latter half of the equation that just uses the magnetic field (electrostatic field calculation are computational expensive and require more clever calculation and programming techniques to render in real-time…I’ll get to this later . Its hard to see in these images, but I used the camera distance from the particle to calculate its opacity, this simulates fog and gives the images better depth.

]]> http://www.syedrezaali.com/blog/?feed=rss2&p=1680 0 http://www.syedrezaali.com/blog/?p=1667 http://www.syedrezaali.com/blog/?p=1667#comments Sun, 07 Nov 2010 19:48:16 +0000 Reza http://www.syedrezaali.com/blog/?p=1667

I got the opportunity to work on a real-time video installation in NYC taking place in the Mixed Greens Gallery. The installation, LoveHateNow, is collaboration between artist, Alexander Reyna and creative technologist Heidi Hysell. I helped with the design of the visual elements, coding, real-time graphics development and visualization of the real-time data streams. If you are in New York November 18, 2010–January 8, 2011, you should definitely check this installation out:

Opening: Mixed Greens Gallery, NYC, Saturday, November 20, 6-8pm

“LoveHateNow is a constantly evolving, real-time, mixed media triptych exploring the mood of New York City through live internet feeds…Viewers will witness a constantly changing landscape of numbers, words, and images projected onto an algorithmically generated triptych…the piece streams information from the The New York Times, the local weather report, Twitter, and other sources. The result is a snapshot that changes based on the minute-to-minute information it receives.”