Kneron PLUS inference APIs

The inference functions provide sophisticated functionally for different applications. Different set of inference APIs would need different models to make it work.

Include Header File: kp_inference.h

Functions

Functions

kp_customized_command_send

send a user-defined command and receive the command result, users also need to implement code in firmware side as well.

int kp_customized_command_send(
    kp_device_group_t devices
    void *cmd
    int cmd_size
    void *return_buf
    int return_buf_size
)

Parameters:

devices         [in]      a set of devices handle.
cmd             [in]      user-defined command buffer, shoud include 'kp_inference_header_stamp_t' in the beginning; using 'job_id' as user-defined command ID, others will be handled by API.
cmd_size        [in]      command buffer size.
return_buf      [out]     user-defined command return buffer in any user-defined format.
return_buf_size [out]     return buffer size.

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_customized_inference_receive

receive inference result of age gender

int kp_customized_inference_receive(
    kp_device_group_t devices
    void *result_buffer
    int buf_size
    int *recv_size
)

Parameters:

devices         [in]      a set of devices handle.
result_buffer   [out]     user-prepared result buffer, when receiving data, it begins with 'kp_inference_header_stamp_t'. user should guarantee buffer size is big enough.
buf_size        [in]      result buffer size.
recv_size       [out]     received result size.

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_customized_inference_send

send image for age gender inference

int kp_customized_inference_send(
    kp_device_group_t devices
    void *header
    int header_size
    uint8_t *image
    int image_size
)

Parameters:

devices         [in]      a set of devices handle.
header          [in]      user-defined image header, shoud include 'kp_inference_header_stamp_t' in the beginning; in the header stamp, only 'job_id' is needed for user to fill in, others will be handled by API.
header_size     [in]      image header size.
image           [in]      image buffer.
image_size      [in]      image buffer size.

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_dbg_receive_checkpoint_data

To receive debug checkpoint data, use it only if you enable kp_dbg_set_enable_checkpoints().

int kp_dbg_receive_checkpoint_data(
    kp_device_group_t devices
    void **checkpoint_buf
)

Parameters:

devices         [in]      a set of devices handle.
checkpoint_buf  [out]     a buffer contains checkpoint data, memory is allocated automatically while needed.

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_dbg_set_enable_checkpoints

Enable/Disable inference breakpoints in firmware for inference debugging purpose.

int kp_dbg_set_enable_checkpoints(
    kp_device_group_t devices
    uint32_t checkpoint_flags
    bool enable
)

Parameters:

devices         [in]      a set of devices handle.
checkpoint_flags[in]      bit-fields settings, refer to kp_dbg_checkpoint_flag_t.
enable          [in]      set enable/disable.

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_generic_data_inference_receive

int kp_generic_data_inference_receive(
    kp_device_group_t devices
    kp_generic_data_inference_result_header_t *output_desc
    uint8_t *raw_out_buffer
    uint32_t buf_size
)

When a image inference is done, this function can be used to get the results in RAW format.

Note that the data received is in Kneron RAW format, users need kp_generic_inference_retrieve_float_node() to convert RAW format data to floating-point data.

Parameters:

devices         [in]      a set of devices handle.
output_desc     [in]      refer to kp_generic_data_inference_result_header_t for describing some information of received data.
raw_out_buffer  [out]     a user-allocated buffer for receiving the RAW data results, the needed buffer size can be known from the 'max_raw_out_size' in 'model_desc' through kp_load_model().
buf_size        [in]      size of raw_out_buffer.

Returns:

int refer to KP_API_RETURN_CODE in kp_struct.h

kp_generic_data_inference_send

Generic raw inference with multiple input images and bypass pre-process send.

int kp_generic_data_inference_send(
    kp_device_group_t devices
    kp_generic_data_inference_desc_t *inf_data
)

This is to perform one model inference with multiple input images without pre-processing on device, it is non-blocking if device buffer queue is not full.

When this is performed, user can issue kp_generic_data_inference_receive() to get the result.

In addition, to have better performance, users can issue multiple kp_generic_data_inference_send() then start to receive results through kp_generic_data_inference_receive().

Parameters:

devices         [in]      a set of devices handle.
inf_data        [in]      inference data of needed parameters for performing inference including image buffer size, model id.

Returns:

int refer to KP_API_RETURN_CODE in kp_struct.h

kp_generic_image_inference_receive

int kp_generic_image_inference_receive(
    kp_device_group_t devices
    kp_generic_image_inference_result_header_t *output_desc
    uint8_t *raw_out_buffer
    uint32_t buf_size
)

Parameters:

devices         [in]      a set of devices handle.
output_desc     [in]      refer to kp_generic_image_inference_result_header_t for describing some information of received data.
raw_out_buffer  [out]     a user-allocated buffer for receiving the RAW data results, the needed buffer size can be known from the 'max_raw_out_size' in 'model_desc' through kp_load_model().
buf_size        [in]      size of raw_out_buffer.

Returns:

int refer to KP_API_RETURN_CODE in kp_struct.h

kp_generic_image_inference_send

Generic raw inference with multiple input images send.

int kp_generic_image_inference_send(
    kp_device_group_t devices
    kp_generic_image_inference_desc_t *inf_data
)

This is to perform one model inference with multiple input images, it is non-blocking if device buffer queue is not full.

When this is performed, user can issue kp_generic_image_inference_receive() to get the result.

In addition, to have better performance, users can issue multiple kp_generic_image_inference_send() then start to receive results through kp_generic_image_inference_receive().

Parameters:

devices         [in]      a set of devices handle.
inf_data        [in]      inference data of needed parameters for performing inference including image buffer size, model id.

Returns:

int refer to KP_API_RETURN_CODE in kp_struct.h

kp_generic_inference_retrieve_fixed_node

Retrieve single node output data from raw output buffer.

kp_inf_fixed_node_output_t *kp_generic_inference_retrieve_fixed_node(
    uint32_t node_idx
    uint8_t *raw_out_buffer
    kp_channel_ordering_t ordering
)

This function retrieves and converts RAW format data to fixed-point data on the per-node basis.

Parameters:

node_idx        [in]      wanted output node index, starts from 0. Number of total output nodes can be known from 'kp_generic_raw_result_header_t'
raw_out_buffer  [in]      the RAW output buffer, it should come from kp_generic_raw_inference_receive().
ordering        [in]      the RAW output channel ordering

Returns:

refer to kp_inf_fixed_node_output_t. It describes fixed-point values of this node in specific channel ordering.

kp_generic_inference_retrieve_float_node

Retrieve single node output data from raw output buffer.

kp_inf_float_node_output_t *kp_generic_inference_retrieve_float_node(
    uint32_t node_idx
    uint8_t *raw_out_buffer
    kp_channel_ordering_t ordering
)

This function retrieves and converts RAW format data to floating-point data on the per-node basis.

Parameters:

node_idx        [in]      wanted output node index, starts from 0. Number of total output nodes can be known from 'kp_generic_raw_result_header_t'
raw_out_buffer  [in]      the RAW output buffer, it should come from kp_generic_raw_inference_receive().
ordering        [in]      the RAW output channel ordering

Returns:

refer to kp_inf_float_node_output_t. It describes floating-point values of this node in specific channel ordering.

kp_generic_inference_retrieve_raw_fixed_node

Retrieve single node output data from raw output buffer.

kp_inf_raw_fixed_node_output_t *kp_generic_inference_retrieve_raw_fixed_node(
    uint32_t node_idx
    uint8_t *raw_out_buffer
)

This function retrieves RAW format data in fixed-point format on the per-node basis.

The return pointer of 'kp_inf_raw_fixed_node_output_t' actually points to raw_out_buffer so do not free raw_out_buffer before completing the use of 'kp_inf_raw_fixed_node_output_t *'

Parameters:

node_idx        [in]      wanted output node index, starts from 0. Number of total output nodes can be known from 'kp_generic_raw_result_header_t'
raw_out_buffer  [in]      the RAW output buffer, it should come from kp_generic_raw_inference_receive().

Returns:

refer to kp_inf_raw_fixed_node_output_t. It describes fixed-point values of this node with the Kneron device origin raw data buffer and channel ordering (KL520: height x channel x width (aligned to 16 byte), KL720: channel x height x width (aligned to 16 byte)).

kp_inference_configure

Configure inference settings.

int kp_inference_configure(
    kp_device_group_t devices
    kp_inf_configuration_t *conf
)

Parameters:

conf            [in]      refer to kp_inf_configuration_t.

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_profile_get_statistics

Collect inference profile results.

int kp_profile_get_statistics(
    kp_device_group_t devices
    kp_profile_data_t *profile_data
)

Parameters:

devices         [in]      a set of devices handle.
profile_data    [out]     refer to kp_profile_data_t.

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_profile_set_enable

To set enable/disable debug profile.

int kp_profile_set_enable(
    kp_device_group_t devices
    bool enable
)

Parameters:

devices         [in]      a set of devices handle.
enable          [in]      set enable/disable.

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_release_dbg_checkpoint_data

Release the debug checkpoint data

int kp_release_dbg_checkpoint_data(
    void *checkpoint_buf
)

Parameters:

checkpoint_buf            checkpoint buffer to be released

Returns:

refer to KP_API_RETURN_CODE in kp_struct.h

kp_release_fixed_node_output

Release the single node output data from raw output buffer

void kp_release_fixed_node_output(
    kp_inf_fixed_node_output_t *fixed_node_output
)

Parameters:

fixed_node_output          kp_inf_fixed_node_output_t to be released

kp_release_float_node_output

Release the single node output data from raw output buffer

void kp_release_float_node_output(
    kp_inf_float_node_output_t *float_node_output
)

Parameters:

fixed_node_output          kp_inf_float_node_output_t to be released

kp_release_raw_fixed_node_output

Release the single node output data from raw output buffer

void kp_release_raw_fixed_node_output(
    kp_inf_raw_fixed_node_output_t *raw_fixed_node_output
)

Parameters:

raw_fixed_node_output          kp_inf_raw_fixed_node_output_t to be released