Video Signal Notes

Once upon a time, I started doing live code visuals, and I made images with a single computer. I would perform by outputting these images to a projector with an HDMI cable. And all was well.

Then I went off to a residency at Alfred University, and learned about the wider world of video and video synthesis. I became obsessed with framebuffers. I bought a vintage synth. I bought a new Eurorack framebuffer that I hoped could stand in for the vintage one, separated as we were by an ocean and a pandemic.

And here I fell into a confusing world of signal types, conversion boxes, cables, and much less documentation than you would find for the average Javascript library. Years into slowly unraveling asides in gear videos and setting up various toolchains on my dining room table, it struck me how much I (& my trusty studio assistant and man about home) have figured out, chunk by chunk.

This set of notes captures the chunks, both for myself and for anyone else who might be interested.

If you see something that's wrong, please tell me; this is essentially the knowledge I have cobbled together as I try to solve various problems and sate various desires, so it is not authoritative in any way. But maybe it's helpful!

These notes cover:

Three Classes of Signal

The base issue in working with video, that which will haunt you always, is the question of working with and converting between the three primary classes of signal: analog, digital, and Eurorack.

Analog

Analog signals are the original video signals — these are the formats used in old CRTs and over-the-air TV signals and mimicked digitally for new televisions.

They are a series of voltages that give instructions to an electron gun to output specific values line after line and draw an image on your television screen. They also contain instructions for when to pause, move the gun, and start drawing a new line or a new frame. These instructions differ geographically, between NTSC (North America, parts of South America), PAL (Europe, other parts of South America), and SECAM (parts of Europe and Africa).

Most old gear you come across will probably be NTSC or PAL. Newer analog gear often handles both. I have not come across SECAM really.

Because the wave form encodes so much data, keeping various devices reading with the same timing (synced) can be an ongoing concern in analog systems. When broken, it can make fun glitches or totally illegible ones. You may find yourself needing a sync generator or otherwise needing to distribute sync pulses through an analog system. If images are sliding all over the place or otherwise not lining up, think sync.

Analog systems usually encode signal as composite or component. Composite encodes all the needed information on one channel, and is usually transported on a cable with an RCA plug, colored yellow. (If you think of red/white/yellow A/V cords, the yellow is the video usually and the red and white the audio channels. Also did you know you can just pull the combined cables apart if you only want one. It was a long time before I did.)

Component commonly refers to YPbPr signals carried on three cables, and that is mostly how I have encountered it. It can refer to RGB component as well, however, so there may be cases where they need to be distinguished.

Finally, there is also S-video, which uses one cable, but has separate pins and is therefore a strange composite component hybrid.

Further Info

Shout out to How Video Works for being a great introduction to the basics of video, without which I could not have started figuring any of the rest of this out.

Understanding Analog Video Signals and Video Basics from Analog Devices have detailed tables and explanations of various signal types.

Digital

Compared to analog, digital signal is relatively simple to work with and comes in two primary formats: HDMI & SDI. HDMI is what comes out of your laptop and carries both audio and video. SDI is a digital format that can have lower resolution than HDMI. It is the lingua franca of the signal converter ecosystem.

Eurorack

Eurorack is an analog signal format that is used by LZX modules and other in the Eurorack video synth system like Syntonie. It is 1V RGB.

In & Out: Segments of a Toolchain

The purpose any video toolchain is to take one kind of signal and make it another kind — connect your computer to a Eurorack system to a projector; or connect a camera to vintage synthesizers or mixers, into your computer to stream or record. This section focuses on each end and the next will talk about the connections going on in the magic cloud.

illustration of inputs and outputs connected by a cloud labelled "connections"

Inputs

Computer

A computer is in some ways the most flexible input, as one can generate images using any number of digital tools. Sending signal from my laptop running La Habra into a system was my first goal.

Luckily this is the most complicated input, so it spurred me to learn this whole system. Usually using computer-based input will require transformation from HDMI to SDI and then onto analog.

Camera

A camera, digital or analog, is a useful input source both on its own or as a secondary method for getting code-based images into a video chain. An analog camera pointed at a laptop screen can replace a number of signal converters and lend a softness to the image that's pretty much impossible to generate with a shader.

Signal Generator

If one is using a Eurorack setup, a signal generator is a way to create images purely from electricity, using ramps and oscillators.

These include inclusive but hard-to-get LZX base modules, like the Vidiot or a Chromagnon, or the more barebones LZX Cadet IV, DSG3 or VU009. (Linked items are currently available; unlinked items are either discontinued or in pre-order.)

Outputs

Projectors

Projectors are useful when you want to show people in the same place what you are doing, especially very big. They are usually very flexible, accepting all signal formats except pure Eurorack.

Monitors/TVs

Another excellent option for showing people in the same room as you your work or monitoring signal is a CRT or digital TV or monitor.

I don't have a lot to say about digital monitors except that I know newer ones can be less willing to display glitchy signals.

CRTs are useful if you are using old synths or mixers and want something that will take analog signal without needing to be stepped up to SDI. They are also useful to monitor this section of your video chain and help debug anything that fails. While they no longer tend to be found in secondhand shops, they are often still listed on Ebay or Craiglist, especially if you are willing to pick them up.

Capture Devices

Should you want to share your video signals into the future or out of your room, you may be interested in a capture device, which I break into standalones and capture cards.

A capture card is a way to transform a digital video signal into something that a computer can read in order to record, stream, or further process. They are often pitched at gamers, although the rise of the Youtube prosumer market means more options at more price points are available. I was reasonably happy with my Elgato GameCapture HD60 — although it felt like Macs were only somewhat grudgingly supported by the card's accompanying software. When paired with OBS Link, which transformed the GameCapture's input into NDI that could be piped into OBS or VDMX, this was a good way to take signal from one computer into another.

As might be apparent from the previous sentence, do be sure to check in which format the card's signal will appear. Now that I've upgraded to the Atem Mini Pro, the signal appears as a webcam input, which makes it rather easier to share with other applications or to stream. (The Atem Mini Pro actually also streams directly from the box, which is a bonus if you want to stream often, because it removes one form of strain from your computer.)

A standalone capture device is something like the Hyperdeck Studio, which has a small monitor and records input signal to SD cards before passing the signal through to a projector or other device. It also takes in a variety of video formats and displays what it is interpreting the signal as, which is useful for debugging your chain.

Connecting Things

Okay, and now for the magic cloud section of the toolchain graph — conversions. Unless your toolchain is purely one sort of signal, you'll need to convert signals, and this is often done one step at a time.

Graph showing the main kinds of signal conversions HDMI ↔ SDI ↔ Analog (Component, Composite) ↔ Eurorack

For the first three signals, one can buy conversion boxes — usually from Blackmagic who have the more affordable end of the market covered. (Yes, sadly, costs only go up from here.) They usually go step-by-step.

HDMI ↔ SDI

This HDMI to SDI 3G micro converter is a great little box for the first step. It will be necessary for nearly all setups with computer input.

Blackmagic also makes the reverse SDI to HDMI and bidirectional boxes. This might be necessary in a setup with the Atem Mini, for instance, where the output would need to be stepped up, but not with a projector or Hyperdeck which take other inputs.

SDI ↔ Analog

When working with an HDMI signal into gear that takes analog input, this will be the next step after conversion to SDI.

I use this box for SDI to Analog and this one for the reverse. Both can convert to/from NTSC or PAL composite and component signals, with Betacam or SMPTE levels, and with or without an 7.5 IRE pedestal. What do the last two mean? I don't know entirely — I do know that the pedestal is associated with NTSC. Overall, it does mean that the digital ↔ analog conversion is flexible enough to work with many generations and types of video gear. These are the workhorses of the processing chain, especially when one wants to work with Eurorack video synthesis.

Often signals from analog cameras come in a composite format and the next step might be looking for them as component. In that case, this pair of boxes can be used to take a composite input, transform it into SDI, and then transform it out as component.

Analog ↔ Eurorack

The final relation is taking an analog composite or component signal and turning it into a 1V RGB signal. This is the format LZX and other Eurorack modules work with (a few, like Syntonie's CBV001 and CBV002, may accept composite).

For this, I use the Syntonie VU003B, paired with the VU007B for sync generation.

(In theory, LZX's TBC2 will cover both this step and the previous — taking SDI signals in and putting out 1V RGB, plus color control, but it's been "shipping soon" for the past two-and-a-half years, so who knows if it will actually exist in anyone's home soon.)

From here, the signal can be patched into further LZX, Syntonie, or other modules, and then converted back to analog signals, which can be plugged directly into a projector or stepped back up into SDI and connected to the Hyperdeck or other capture device that takes SDI. In my setup, either the VU007B or Memory Palace can do this work.

A Note: Plug Types Are Not Signal Types, Mostly

When I first started diving into these conversions, I would get confused between plug type and signal type; this makes sense: in some cases — RCA, BNC — they are not tightly linked, while in others — HDMI, S-Video — they are. And so we arrive in a situation where analog composite and component signals are often carried by cables with RCA ends, while SDI signals are often carried by cables with BNC ends, but simply plugging a BNC to RCA cable into a composite input at one end and an SDI output at the other will not convert the signal type.

It is vital therefore to distinguish the two in one's mind when setting up a toolchain.

These are the input types you are likely to see and the types of signal they are often used for:

image name signal/device
3.5 mm eurorack
bnc sdi or analog, common to the converter system and some analog cameras
hdmi HDMI (well that one is simple)
rca analog composite or component
s-video analog s-video
usb-c HDMI, power
vga analog component or HDMI (with adapter)

In my setup, I get lots of use out of BNC cables, RCA cables, BNC to RCA, plus some HDMI and USB-C. USB-C converter docks and USB-C to HDMI adapters are also very useful, and an S-Video cable might come in handy for vintage gear.

Two Example Setups

Finally, here are two example setups I've put together with different analog tools. (I have another writeup with purely digital setups.) I hope they help answer some questions as well as inspire you to work on your own toolchain. It can take a lot of patience (as well as stalking secondhand sites for deals), but it's worth it in the end!

Fairlight Setup + Stream

fairlight setup

Memory Palace Setup

Note: This shows multiple options for output. Only one output to the projector is necessary, for instance.

mem pal setup

Bonus Fun

These notes are specific to my setup and things I want to do. If you are interested in the other kinds of things that are possible with video mixers and Eurorack synths, these videos might get you excited:

↑