Network protocols for VMS
1. Introduction
This agreement specifies the communication method and protocol between the AI end, the control end, and the pod. Note that the pods referred to in this agreement specifically refer to Tofu type pods, and other pods and PTZ are not applicable to this communication method.
When using non Windows systems such as Android and Linux on the control end, this JSON protocol can be used. This protocol has no platform restrictions and can be used for secondary development of client software. Both SDK and this protocol can be used on the Windows side.
Figure 1 Communication Relationship
During the operation process, the AI end directly controls the pan tilt and lens, with the AI end serving as the server and the pan tilt and lens control end serving as the client, with a port number of 8090. Adopting PelcoD protocol and VISCA protocol, hereinafter referred to as PV protocol. The default control and query cycle is once every 200ms.
Support algorithm mode switching and parameter configuration on the AI side in the control end; Information reception during the identification and tracking process; The manual control of the pan tilt and lens, with the AI end serving as the server and the control end serving as the client, has a port number of 8089. Adopting JSON protocol. The frame spacing between all JSON data should not be less than 300 milliseconds。
The AI end uses the RTSP protocol to transmit videos, with H.264 encoding by default and port number 554. When the AI camera node is turned on, start the RTSP server and wait for video on demand. When the terminal needs to view the video, the video can be on-demand through the RTSP URL format.
All devices connected to the AI end in the network link must comply with heartbeat frame rules.
2. PV communication protocol
The AI end communicates with the pod using the PV protocol, with PelcoD protocol for pan tilt motion and query related commands, and VISCA protocol for lens zoom, focus, and query related commands.
Communication does not include data other than other instructions, and can be sent and received directly through the network port. Here are a few commonly used protocol instructions, Please refer to the complete agreement 《11. Non continuously variable PelcoD pan tilt control and VISCA lens control protocol (reference only)》document。
3. JSON Communication protocol
The JSON communication protocol is suitable for communication between clients and AI, used to control and configure the AI end, obtain process messages from the AI end, and manually control the pan tilt and lens.
In the following data frame description, S represents Server (AI module) and C represents Client.
Table 1 Protocol Format
Identification code
Frame type
Frame length
Frame Content
1Byte
1Byte
1Byte
4Byte
N
0xEC
0x91
See frame type table
Frame payload length
JSON
Image capture protocol
Identification code
Frame type
Frame length
Position
Frame Content
Frame check
Frame end identification code
1Byte
1Byte
1Byte
4Byte
8Byte
N
1Byte
1Byte
1Byte
0xEB
0x92
0x04
Left
Top
Width
Height
JPEG
Sum of the first 7 bytes
0xFB
0x92
Explanation of Image Capture Protocol：
Image capture is individually packaged for each target. Example: In this image, there are three targets, and the protocol will send three independent protocol packets.
Position: The coordinate position of the target in the original image, including left (2Byte), top (2Byte), width (2Byte), and height (2Byte).
Frame length: The size of the JPEG image.
Frame check：0xEB+0x92+0x04+ frame length。
Table 2 Frame Type Table
Frame type
Type name
Notes
0x01(Device to client)
Status frame
JSON
0x03(client to device)
Control command
JSON
0x04
Image capture
JPEG
0x05
Image information query
0x06
Set target detection area
JSON
0x07(client to device)
Set Display Mode
JSON
0x08(client to device)
Set Algorithm Model
JSON
0x11
Heartbeat
0x12
ACK
4. JSON frame content description
4.1 Server to client frame content
AI operation process information push
Key
name
Value
type
Notes
ControlType
control type
AIInfo
String
WorkMode
Working mode
0x01：identify
0x02：auto tracking
0x03：click to track
0x04：gate tracking
Int
ObjectCount
Total number of targets
N
Int
Object
Class
target type
identify
Person:0xA1
Int
Car:0xA2
track
Normal:0xB1
Loss:0xB2
Points
position
Left
X
Int
Position of the target box，
Zero point in the upper left corner (0.0),
Value range：
visible light：
Left、Right 0-1919
Top、Bottom 0-1079
infrared：
Left、Right 0-719
Top、Bottom 0-575；
When tracking status, only push the coordinates of the tracking target，Coordinates are all 0 0，0，0,when tracking is lost，
Top
Y
Right
X
Bottom
Y
JSON Example：
{
"ControlType": "AIInfo",
"WorkMode": 1,
"ObjectCount": 2,
"Object": {
"01": {
"Class": 161,
"Points": {
"Left": 100,
"Top": 100,
"Right": 200,
"Bottom": 200
}
},
"02": {
"Class": 162,
"Points": {
"Left": 300,
"Top": 100,
"Right": 400,
"Bottom": 200
}
}
}
}
JSON example during tracking process：
{
"ControlType": "AIInfo",
"WorkMode": 2,
"ObjectCount": 1,
"Object": {
"01": {
"Class": 177,
"Points": {
"Left": 220,
"Top": 450,
"Right": 250,
"Bottom": 490
}
},
}
}
Status frame: Multiplication and pan tilt angle push
Key
Name
Value
Type
Notes
ControlType
control type
ZoomInfo
String
ZoomInfo
Zoom
Accurate to one decimal place
Double
PTZInfoH
Pan angle
Accurate to one decimal place
Double
PTZInfoV
Tilt angle
Accurate to one decimal place
Double
CamShowMode
Camera display mode
0: color
1: black and white
Int
JSON Example：
{
"ControlType": "ZoomInfo",
"ZoomInfo": 3.6,
"PTZInfoH": -50.3,
"PTZInfoV": 16.4,
"CamShowMode":0
}
4.2 Client to server frame content
Control command: working mode switching
Key
Name
Value
Type
Notes
ControlType
control type
SetWorkMode
String
SetWorkMode
Working mode
0x00: close AI
0x01：identify
0x02：auto tracking
0x03：Click to track
0x04：gate tracking
0x05：Auto Zoom On
0x06：Auto zoom off
Int
P2Track
Center
X
abscissa
Int
top-left 1,1
Y
ordinate
Int
Distance
Search range in gate tracking mode
Int
Search for square dimensions；
Omit this key value when the search scope is rectangular.
DistanceX
Horizontal search range in gate tracking mode
Int
Search rectangle width；
When the search range is a square, omit this key value.
DistanceY
Longitudinal search range in gate tracking mode
Int
Search rectangle height；
Omit this key value when the search range is square.
JSON Example：
{
"ControlType": "SetWorkMode",
"SetWorkMode": 1
}
The information P2Track is only input in point tracking mode and gate tracking mode, providing the center point coordinates and search range of the target.
JSON Example（Click to track, search within a square）：
{
"ControlType": "SetWorkMode",
"SetWorkMode": 3
“P2Track”:{
“Center”:{
“X”:310,
“Y”:660
}
“Distance”:30
}
}
JSON Example（Gate tracking, search range is rectangular）：
{
"ControlType": "SetWorkMode",
"SetWorkMode": 3
“P2Track”:{
“Center”:{
“X”:310,
“Y”:660
}
“DistanceX”:40,
“DistanceX”:50
}
}
Control command：transparent transmission PELCO-D
Key
Name
Value
Type
Notes
ControlType
Control type
SerialControl
String
SerialType
Serial port type
PELCO_D
String
SerialData
Lens
Length
N
Int
Data
Data
N
String
JSON Example：
{
"ControlType": "SerialControl",
"SerialType": "PELCO_D",
"SerialData": {
"Lens": 7,
"Data": "FF01000800FF08"
}
}
4.3 Client Query Image Parameter Information
Customer frame sending format：
Identification code
Frame type
Frame length
1Byte
1Byte
1Byte
4Byte
0xEC
0x91
0x05
0x00 00 00 00
Query information feedback：
Key
Name
Value
Type
Notes
ControlType
control type
ImageSetting
String
ImageSize
Resolution
0:1080P
1:720P
int
ImageBit
Image bitrate
512~4096
Int
ImageCode
Encoding format
0:H264
1:H265
int
JSON Example：
{
"ControlType": " ImageSetting",
" ImageSize ": 0,
" ImageBit ": 4096,
" ImageCode ": 0
}
4.4 Client Setting Target Detection Area
Set target detection area
Key
Name
Value
Type
Notes
ControlType
Control type
SetAreaDot
String
WarnArea
AreaItem
zoning
0: not used
1: used
Int
AreaPoint
Area corner
X
abscissa
Int
1、Coordinates of the upper left corner of the screen（1,1）；
2、AreaPoint Up to 6
Y
ordinate
Int
JSON Example：
1. Set detection area：
{
"ControlType": "SetAreaDot",
"WarnArea": {
"AreaItem": 1,
"AreaPoint": [
{
"X": 120,
"Y": 100
}, {
"X": 310,
"Y": 100
}, {
"X": 310,
"Y": 360
}, {
"X": 120,
"Y": 360
}
]
}
}
2. Cancel detection area：
{
"ControlType": "SetAreaDot",
"WarnArea": {
"AreaItem": 0
}
}
4.5 Client Setting Display Mode
Key
Name
Value
Type
Notes
ControlType
Control type
PipShowSetting
String
PipShow
Display type
0: Large visible light，
Small image infrared
1: infrared
2: visible light
3: fuse
16:Small visible light，
Large image infrared
Int
JSON Example：
{
"ControlType": " PipShowSetting ",
"PipShow ": 0
}
4.6 Client Setting Algorithm Model
Key
Name
Value
Type
Notes
ControlType
Control type
ModelSetting
String
Model
Algorithm model
0: visible light
1: infrared
Int
JSON Example：
{
"ControlType": "ModelSetting",
" Model ": 0
}
5. ACK and Heartbeat Frame
5.1 ACK Frame
Both the server and client need to perform an ACK response after receiving the frame. If there is no ACK within 15 seconds, the server will forcibly offline the client, and the client needs to reconnect to the server.
The server actively uploads information, and the client does not need to reply with an ACK.
Identification code
Frame type
Frame length
Status
1Byte
1Byte
1Byte
4Byte
2Byte
0xEC
0x91
0x12
0x00 00 00 02
0x00 XX
Status
Describe
0x00 00
Execute normally
0x00 01
Incomplete package
0x00 02
Protocol content error
5.2 Heartbeat Frame
Identification code
Frame type
Frame length
1Byte
1Byte
1Byte
4Byte
0xEC
0x91
0x11
0x00 00 00 00
Heartbeat packet format
The client needs to send a heartbeat frame to the server within 15 seconds. When the server receives the heartbeat frame from the client, it will reply to the heartbeat within 1 second. If the client does not receive a reply after a timeout, the client needs to resend the frame.
When the server does not receive a heartbeat frame from the client within 15 consecutive seconds, the server will determine that the client has automatically gone offline and stop sending and receiving data.
When the client does not receive a response from the server within 15 consecutive seconds, it should be determined that the connection has been interrupted and needs to be reconnected.
Last updated