Scannertools API


Operations

Face Detection

ops.MTCNNDetectFaces(frame)

Detect human poses in an image. Registered with import scannertools.face_detection.

Stream parameters:

Parameters

frame (scannerpy.types.FrameType) – The frames to detect poses in.

Returns

A list of bounding boxes.

Return type

scannerpy.types.BboxList

import scannerpy as sp
import scannertools.face_detection
import scannertools.vis

def main():
  # Compute face bounding boxes and draw them on a sample video
  with sp.util.sample_video() as video_path:
    cl = sp.Client()
    video = sp.NamedVideoStream(cl, 'example', path=video_path)
    frames = cl.io.Input([video])
    faces = cl.ops.MTCNNDetectFaces(frame=frames)
    drawn_faces = cl.ops.DrawBboxes(frame=frames, bboxes=faces)
    output_video = sp.NamedVideoStream(cl, 'example_faces')
    output_op = cl.io.Output(drawn_faces, [output_video])
    cl.run(output_op, sp.PerfParams.estimate())
    output_video.save_mp4('example_faces')
    # output video is saved to 'example_faces.mp4'

if __name__ == "__main__":
    main()

Face Embedding

ops.EmbedFaces(frame, bboxes)

Compute a face embeddings vector for each bounding box in the image. Registered with import scannertools.face_embedding.

Stream parameters:

Parameters
  • frame (scannerpy.types.FrameType) – The frames to detect poses in.

  • bboxes (scannerpy.types.BboxList) – Bounding boxes to compute face embeddings on.

Returns

Serialized face embeddings.

Return type

bytes

Gender Detection

ops.DetectGender(frame, bboxes)

Detect the gender of people in an image. Registered with import scannertools.gender_detection.

Stream parameters:

Parameters
  • frame (scannerpy.types.FrameType) – The frames to detect gender in.

  • bboxes (scannerpy.types.BboxList) – Bounding boxes to indicate where to estimate gender.

Returns

The detected genders.

Return type

Any

Object Detection

ops.DetectObjects(frame, bboxes)

Detect objects in an image. Registered with import scannertools.object_detection.

Stream parameters:

Parameters

frame (scannerpy.types.FrameType) – The frames to detect objects in.

Returns

A list of bounding boxes for the detected objects.

Return type

scannerpy.types.BboxList

Pose Detection

ops.OpenPose(model_directory, pose_num_scales, pose_scale_gap, compute_hands, hand_num_scales, hand_scale_gap, compute_face, frame)

Detect human poses in an image. Registered with import scannertools_caffe.

Init parameters:

Parameters
  • model_directory (str) – A path to the directory with the OpenPose model files.

  • pose_num_scales (int) – The number of scales to evaluate the pose network on.

  • pose_scales_gap (float) – The scaling factor between scales for the pose network.

  • compute_hands (bool) – Flag to enable computing hand keypoints.

  • hand_num_scales (int) – The number of scales to evaluate the hand network on.

  • hand_scale_gap (float) – The scaling factor between scales for the hand network.

  • compute_face (bool) – Flag to enable computing face keypoints.

Stream parameters:

Parameters

frame (scannerpy.types.FrameType) – The frames to detect poses in.

Returns

A list of detected poses.

Return type

scannertools_caffe.pose_detection.PoseList

Shot Detection

https://storage.googleapis.com/scanner-data/public/sample-shots-small.jpg
ops.ShotBoundaries(histograms)

Detect shot boundaries using color histograms of frames. Registered with import scannertools.shot_detection.

Stream parameters:

Parameters

histograms (scannerpy.types.Histogram) – The color histograms of frames from a video.

Returns

The indices of the shot boundaries.

Return type

Sequence[int]

Resize

ops.Resize(width, height, min, preserve_aspect, interpolation, frame)

Resize images to a fixed size. Registered with import scannertools.imgproc.

Stream config parameters:

Parameters
  • width (Sequence[int]) – The target width of the frame.

  • height (Sequence[int]) – The target height of the frame.

  • min (Sequence[bool]) – If true, resizes frames to width and height only if the input frames width and height if both are less than width and height.

  • preserve_aspect (Sequence[bool]) – If true, sets width to preserve the aspect ratio of the input frame if height is non-zero. Likewise if width is non-zero.

  • interpolation (Sequence[str]) – The type of interpolation to use.

Stream parameters:

Parameters

frame (scannerpy.types.FrameType) – The frame to resize.

Returns

The resized frame.

Return type

scannerpy.types.FrameType

Optical Flow

ops.OpticalFlow(frame)

Computes optical flow from one frame to the next. Registered with import scannertools.imgproc.

Stream parameters:

Parameters

frame (scannerpy.types.FrameType) – The frame to compute flow on.

Returns

The flow field.

Return type

scannerpy.types.FrameType

Histogram

ops.Histogram(frame)

Computes a color histogram of the frame. Registered with import scannertools.imgproc.

Stream parameters:

Parameters

frame (scannerpy.types.FrameType) – The frame to process.

Returns

The color histogram.

Return type

scannerpy.types.Histogram

Tracker

Utilities for writing Operations

Caffe2

Pytorch

TensorFlow

Storage

scannertools_caffe