<Software Download V2.0.1.7>

1. Introduction

This agreement specifies the communication method and protocol between the AI end, the control end, and the pod. Note that the pods referred to in this agreement specifically refer to Tofu type pods, and other pods and PTZ are not applicable to this communication method.

When using non Windows systems such as Android and Linux on the control end, this JSON protocol can be used. This protocol has no platform restrictions and can be used for secondary development of client software. Both SDK and this protocol can be used on the Windows side.

Figure 1 Communication Relationship

During the operation process, the AI end directly controls the pan tilt and lens, with the AI end serving as the server and the pan tilt and lens control end serving as the client, with a port number of 8090. Adopting PelcoD protocol and VISCA protocol, hereinafter referred to as PV protocol. The default control and query cycle is once every 200ms.

Support algorithm mode switching and parameter configuration on the AI side in the control end; Information reception during the identification and tracking process; The manual control of the pan tilt and lens, with the AI end serving as the server and the control end serving as the client, has a port number of 8089. Adopting JSON protocol. The frame spacing between all JSON data should not be less than 300 milliseconds。

The AI end uses the RTSP protocol to transmit videos, with H.264 encoding by default and port number 554. When the AI camera node is turned on, start the RTSP server and wait for video on demand. When the terminal needs to view the video, the video can be on-demand through the RTSP URL format.

All devices connected to the AI end in the network link must comply with heartbeat frame rules.

2. PV communication protocol

The AI end communicates with the pod using the PV protocol, with PelcoD protocol for pan tilt motion and query related commands, and VISCA protocol for lens zoom, focus, and query related commands.

Communication does not include data other than other instructions, and can be sent and received directly through the network port.

3. JSON Communication protocol

The JSON communication protocol is suitable for communication between clients and AI, used to control and configure the AI end, obtain process messages from the AI end, and manually control the pan tilt and lens.

In the following data frame description, S represents Server (AI module) and C represents Client.

Table 1 Protocol Format

Image capture protocol

Identification code		Frame type	Frame length	Position				Frame Content	Frame check	Frame end identification code
1Byte	1Byte	1Byte	4Byte	8Byte				N	1Byte	1Byte	1Byte
0xEB	0x92	0x04		Left	Top	Width	Height	JPEG	Sum of the first 7 bytes	0xFB	0x92

Explanation of Image Capture Protocol：

      Image capture is individually packaged for each target. Example: In this image, there are three targets, and the protocol will send three independent protocol packets.

      Position: The coordinate position of the target in the original image, including left (2Byte), top (2Byte), width (2Byte), and height (2Byte).

      Frame length: The size of the JPEG image.   

Frame check：0xEB+0x92+0x04+ frame length。

     

Table 2 Frame Type Table

4. JSON frame content description

4.1 Server to client frame content

AI operation process information push

 

JSON Example：

{
  "ControlType": "AIInfo",
  "WorkMode": 1,
  "ObjectCount": 2,
  "Object": {
    "01": {
      "Class": 161,
      "Points": {
        "Left": 100,
        "Top": 100,
        "Right": 200,
        "Bottom": 200
      }
    },
    "02": {
      "Class": 162,
      "Points": {
        "Left": 300,
        "Top": 100,
        "Right": 400,
        "Bottom": 200
      }
    }
  }
}

JSON example during tracking process：

{
  "ControlType": "AIInfo",
  "WorkMode": 2,
  "ObjectCount": 1,
  "Object": {
    "01": {
      "Class": 177,
      "Points": {
        "Left": 220,
        "Top": 450,
        "Right": 250,
        "Bottom": 490
      }
    }
  }
}

Status frame: Multiplication and pan tilt angle push

Key	Name	Value	Type	Notes
ControlType	control type	ZoomInfo	String
ZoomInfo	Zoom	Accurate to one decimal place	Double
PTZInfoH	Pan angle	Accurate to one decimal place	Double
PTZInfoV	Tilt angle	Accurate to one decimal place	Double
LaserRange	distance	Accurate to one decimal place	Double
CamShowMode	Camera display mode	0: color 1: mono	Int

JSON Example：

{
  "ControlType": "ZoomInfo",
  "ZoomInfo": 3.6,
  "PTZInfoH": -50.3,
  "PTZInfoV": 16.4,
  "CamShowMode":0
}

4.2 Client to server frame content

Control command: working mode switching

Key	Name	Value		Type	Notes
ControlType	control type	SetWorkMode		String
SetWorkMode	Working mode	0x00: close AI 0x01：detection 0x02：auto   tracking 0x03：Click to   track 0x04：gate tracking 0x05：Auto Zoom On 0x06：Auto zoom off		Int
P2Track	Center	X	image position	Int	top-left 1,1
		Y	image position	Int
	Distance	Search range in gate tracking mode		Int	Search for square dimensions； Omit this key value when the search scope is rectangular.
	DistanceX	Horizontal search range in gate tracking mode		Int	Search rectangle width；When the search range is a square, omit this key value.
	DistanceY	Longitudinal search range in gate tracking mode		Int	Search rectangle height； Omit this key value when the search range is square.

JSON Example：

{
  "ControlType": "SetWorkMode",
  "SetWorkMode": 1
}

    The information P2Track is only input in point tracking mode and gate tracking mode, providing the center point coordinates and search range of the target.

JSON Example（Click to track, search within a square）：

{
"ControlType": "SetWorkMode",
"SetWorkMode": 4,
"P2Track": {
  "Center": {
    "X": 310,
    "Y": 660
  },
  "Distance": 64
  }
}

 

JSON Example（Gate tracking, search range is rectangular）：

{
  "ControlType": "SetWorkMode",
  "SetWorkMode": 3
  “P2Track”:{
    “Center”:{
       “X”:310,
       “Y”:660
     }
   “DistanceX”:40,
   “DistanceY”:50
  }
}

 

Control command：transparent transmission PELCO-D

Key		Name	Value	Type	Notes
ControlType		Control type	SerialControl	String
SerialType		Serial port type	PELCO_D	String	PT control
			VISCA	String	lens control
SerialData	Lens	Length	N	Int
	Data	Data	N	String

JSON Example：

{
  "ControlType": "SerialControl",
  "SerialType": "PELCO_D",
  "SerialData": {
    "Lens": 7,
    "Data": "FF01000800FF08"
  }
}

 

4.3 Client Query Image Parameter Information

Customer frame sending format：

 

Query information feedback：

Key	Name	Value	Type	Notes
ControlType	control type	ImageSetting	String
ImageSize	Resolution	0:1080P 1:720P 2:D1 3:1440P	int
ImageBit	Image bitrate	512~4096	Int
ImageCode	Encoding format	0:H264 1:H265	int

 

JSON example:

{
  "ControlType": " ImageSetting",
  " ImageSize ": 0,
  " ImageBit ": 4096,
  " ImageCode ": 0
}

 

4.4 Client Setting Target Detection Area

Set target detection area

Key		Name	Value		Type	Notes
ControlType		Control type	SetAreaDot		String
WarnArea	AreaItem	AreaItem	0: not used 1: used		Int
	AreaPoint	Area corner point	X	abscissa	Int	1、Coordinates of the upper left corner of the screen（1,1）； 2、AreaPoint Up to 6
			Y	ordinate	Int

 

JSON Example：

1. Set detection area：

{
      "ControlType": "SetAreaDot",
      "WarnArea": {
            "AreaItem": 1,
            "AreaPoint": [
                  {
                       "X": 120,
                       "Y": 100
                  }, {
                       "X": 310,
                       "Y": 100
                  }, {
                       "X": 310,
                       "Y": 360
                  }, {
                       "X": 120,
                       "Y": 360
                  }
            ]
      }
}

2. Cancel detection area：

{
      "ControlType": "SetAreaDot",
      "WarnArea": {
            "AreaItem": 0
      }
}

4.5 Client Setting Display Mode

Key	Name	Value	Type	Notes
ControlType	Control type	PipShowSetting	String
PipShow	Display type	0: Thermal in Visual 1:Thermal 2: Visual 3: Fusion 16:Visual in thermal	Int	Not all devices support this function. Depends on the device type.

JSON Example：

{
    "ControlType": " PipShowSetting ",
    "PipShow ": 0
}

4.6 Client Setting Algorithm Model

Key	Name	Value	Type	Notes
ControlType	Control type	ModelSetting	String
Model	Algorithm model	0:visual mix 1:thermal mix 2:person & car 3:ship 4:drone 5:airplane & helicopter 6:bird	Int

JSON Example：

{
    "ControlType": "ModelSetting",
    " Model": 3
}

 

4.7 Client Setting Image Snap Mode

Key	Name	Value	Type	Notes
ControlType	Control type	ImageUpload	String
Upload	Image Snap Mode	0: Inavtivate 1: Activate	Int

JSON Example：

{
    "ControlType": "ImageUpload",
    "Upload": 1
}

 

5. ACK and Heartbeat Frame

5.1 ACK Frame

Both the server and client need to perform an ACK response after receiving the frame. If there is no ACK within 15 seconds, the server will forcibly offline the client, and the client needs to reconnect to the server.

The server actively uploads information, and the client does not need to reply with an ACK.

 

 

5.2 Heartbeat Frame

Heartbeat packet format

   The client needs to send a heartbeat frame to the server within 15 seconds. When the server receives the heartbeat frame from the client, it will reply to the heartbeat within 1 second. If the client does not receive a reply after a timeout, the client needs to resend the frame.

   When the server does not receive a heartbeat frame from the client within 15 consecutive seconds, the server will determine that the client has automatically gone offline and stop sending and receiving data.

   When the client does not receive a response from the server within 15 consecutive seconds, it should be determined that the connection has been interrupted and needs to be reconnected.