PTZ control by Object Tracking

TheNukem

New Member
Hi all.

I've been a lurker on these forums for a while, but recently I've not been able to sit by and watch.


So this is a call out to someone who has skills far beyond mine....and I can tell you that having read a LOT of posts on these kind of forums, that this is a plugin that people are crying out for.

It'll save $1,000s of dollars on these crappy PTZs that claim to have AI tracking. OBS can do it all.

There are 2 awesome plugins that basically need linking, but I'm not the one to do it. I'm techie AF, but not a coder.

https://github.com/glikely/obs-ptz/
AND
https://github.com/locaal-ai/obs-detect
This is how I see it flowing:
  1. OBS captures video → Object Detection Plugin processes frames.
  2. Detection script calculates object position → sends PTZ adjustments.
  3. PTZ Plugin translates this into VISCA commands → Camera moves.

a few challenges:
  • Latency: PTZ movement may lag behind fast-moving objects.
  • Smoothing: I'll need to implement dead zones (only move if object is >X pixels off-center).
  • Calibration: Fine-tune pan/tilt speeds for smooth tracking.

Assuming that 'detect' already calcs the x, y positioning, then that data needs to feed it into PTZ control with an instruction set along the lines of 'if object is 'z' number of pixels off a specific point, then move camera until object is within tolerance.

Like I say, I'm no coder, but I am an award winning film maker as well as doing lots of multi-cam (PTZs) streaming of live events, performances and conferences. If I can be of any help to you in the development process, please let me know.
 
I’m all for it! obs-ptz already exposes a basic api for control by another plugin, and it is used by @norihiro ‘s Face Tracker plugin . Other plugins could use it too.

The API is a little basic though. I’m thinking about creating a common API that allows other plugins (e.g., @DistroAV ) to expose PTZ controls which can be driven by obs-ptz or other plugins. Object tracking would fit right with that.
 
Second this!
i need this for my local raceclub, i could setup cameras to auto follow cars and then return to preset 1 if none found for 10 seconds or something like that. That would give me more time to focus on commentating!
Please someone make this happen. i will raise money for you if needed!
Detect plugin worked amazing on cars but no PTZ control.
 
SO,

As much as I've loved this idea as it is so far, I've run into another issue.

I have 'proper' PTZ cameras. Panasonic AW HE130s....4 of them and guess what (and I've known this forever)....they run on their own protocol. Not Onvif / Visca / Pelco.

This hasn't been much of a problem for me as I use Companion (BitFocus) to control and preset / recall etc and a touch screen windows machine to do all the quick 'touch (click) to centre on the cursor' kinda things. And that's ALL good...

BUT this isn't going to work so easily with OBS PTZ Control, because OBS PTZ Control....doesn't control /my/ PTZs. :-(
 
How far with this? Has anyone been able to achieve it yet ? This plugin is exactly what I need
I literately made progress on this today. I published a new draft api that is implemented in PTZ Controls and an object tracker can use to control a camera. It can be found here:


That’s only the camera API side though. It still needs a controller to use it.
 
SO,

As much as I've loved this idea as it is so far, I've run into another issue.

I have 'proper' PTZ cameras. Panasonic AW HE130s....4 of them and guess what (and I've known this forever)....they run on their own protocol. Not Onvif / Visca / Pelco.

This hasn't been much of a problem for me as I use Companion (BitFocus) to control and preset / recall etc and a touch screen windows machine to do all the quick 'touch (click) to centre on the cursor' kinda things. And that's ALL good...

BUT this isn't going to work so easily with OBS PTZ Control, because OBS PTZ Control....doesn't control /my/ PTZs. :-(
What does “Proper” mean in terms of control protocol? Surely they talk some form of standard protocol if they integrate with other control systems. If you know the protocol it is possible to add it to the PTZ Controls plugin.
 
Hi all, I'm brand new to the forums but registered to I can chime in on this thread. It's a very cool idea!

I've been looking for a budget-friendly way to record my band in our home recording studio when we play together. I recently purchased a set of 3 1080p PTZ cameras (that support basic UVC controls for PTZ) for dirt cheap to upgrade from the cheap webcams I've been using. I have the OBS plugin "Advanced Scene Switcher" installed, which can detect when certain audio inputs exceed a threshold and switches to that person's camera when they begin playing or singing to give them the camera automatically. I would LOVE to be able to use something like what OP suggested to track and move the cameras as well to follow the action. Someone might be singing for a bit and I'd want to punch in, but if they pull away from the mic and start playing guitar, it would be cool to zoom out to not miss anything. This would be a dream come true and would basically let us have automatic music videos without someone manning the PTZ controls.

I'm a software developer by day and musician by night. My area of expertise is more web focused, so I can only really read C++ (and not write it) or else I'd try to dive in myself. If there's any testing or feedback I can provide to help this project along, I'm ready and willing.
 
Back
Top