Kneron PLUS inference APIs
The inference functions provide sophisticated functionally for different applications. Different set of inference APIs would need different models to make it work.
Include Header File: kp_inference.h
- Functions
- kp_customized_command_send
- kp_customized_inference_receive
- kp_customized_inference_send
- kp_dbg_receive_checkpoint_data
- kp_dbg_set_enable_checkpoints
- kp_generic_data_inference_receive
- kp_generic_data_inference_send
- kp_generic_image_inference_receive
- kp_generic_image_inference_send
- kp_generic_inference_retrieve_fixed_node
- kp_generic_inference_retrieve_float_node
- kp_generic_inference_retrieve_raw_fixed_node
- kp_inference_configure
- kp_profile_get_statistics
- kp_profile_set_enable
- kp_release_dbg_checkpoint_data
- kp_release_fixed_node_output
- kp_release_float_node_output
- kp_release_raw_fixed_node_output
Functions
kp_customized_command_send
send a user-defined command and receive the command result, users also need to implement code in firmware side as well.
int kp_customized_command_send(
kp_device_group_t devices
void *cmd
int cmd_size
void *return_buf
int return_buf_size
)
Parameters:
devices [in] a set of devices handle. cmd [in] user-defined command buffer, shoud include 'kp_inference_header_stamp_t' in the beginning; using 'job_id' as user-defined command ID, others will be handled by API. cmd_size [in] command buffer size. return_buf [out] user-defined command return buffer in any user-defined format. return_buf_size [out] return buffer size.
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_customized_inference_receive
receive inference result of age gender
int kp_customized_inference_receive(
kp_device_group_t devices
void *result_buffer
int buf_size
int *recv_size
)
Parameters:
devices [in] a set of devices handle. result_buffer [out] user-prepared result buffer, when receiving data, it begins with 'kp_inference_header_stamp_t'. user should guarantee buffer size is big enough. buf_size [in] result buffer size. recv_size [out] received result size.
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_customized_inference_send
send image for age gender inference
int kp_customized_inference_send(
kp_device_group_t devices
void *header
int header_size
uint8_t *image
int image_size
)
Parameters:
devices [in] a set of devices handle. header [in] user-defined image header, shoud include 'kp_inference_header_stamp_t' in the beginning; in the header stamp, only 'job_id' is needed for user to fill in, others will be handled by API. header_size [in] image header size. image [in] image buffer. image_size [in] image buffer size.
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_dbg_receive_checkpoint_data
To receive debug checkpoint data, use it only if you enable kp_dbg_set_enable_checkpoints().
int kp_dbg_receive_checkpoint_data(
kp_device_group_t devices
void **checkpoint_buf
)
Parameters:
devices [in] a set of devices handle. checkpoint_buf [out] a buffer contains checkpoint data, memory is allocated automatically while needed.
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_dbg_set_enable_checkpoints
Enable/Disable inference breakpoints in firmware for inference debugging purpose.
int kp_dbg_set_enable_checkpoints(
kp_device_group_t devices
uint32_t checkpoint_flags
bool enable
)
Parameters:
devices [in] a set of devices handle. checkpoint_flags[in] bit-fields settings, refer to kp_dbg_checkpoint_flag_t. enable [in] set enable/disable.
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_generic_data_inference_receive
int kp_generic_data_inference_receive(
kp_device_group_t devices
kp_generic_data_inference_result_header_t *output_desc
uint8_t *raw_out_buffer
uint32_t buf_size
)
When a image inference is done, this function can be used to get the results in RAW format.
Note that the data received is in Kneron RAW format, users need kp_generic_inference_retrieve_float_node() to convert RAW format data to floating-point data.
Parameters:
devices [in] a set of devices handle. output_desc [in] refer to kp_generic_data_inference_result_header_t for describing some information of received data. raw_out_buffer [out] a user-allocated buffer for receiving the RAW data results, the needed buffer size can be known from the 'max_raw_out_size' in 'model_desc' through kp_load_model(). buf_size [in] size of raw_out_buffer.
Returns:
int refer to KP_API_RETURN_CODE in kp_struct.h
kp_generic_data_inference_send
Generic raw inference with multiple input images and bypass pre-process send.
int kp_generic_data_inference_send(
kp_device_group_t devices
kp_generic_data_inference_desc_t *inf_data
)
This is to perform one model inference with multiple input images without pre-processing on device, it is non-blocking if device buffer queue is not full.
When this is performed, user can issue kp_generic_data_inference_receive() to get the result.
In addition, to have better performance, users can issue multiple kp_generic_data_inference_send() then start to receive results through kp_generic_data_inference_receive().
Parameters:
devices [in] a set of devices handle. inf_data [in] inference data of needed parameters for performing inference including image buffer size, model id.
Returns:
int refer to KP_API_RETURN_CODE in kp_struct.h
kp_generic_image_inference_receive
int kp_generic_image_inference_receive(
kp_device_group_t devices
kp_generic_image_inference_result_header_t *output_desc
uint8_t *raw_out_buffer
uint32_t buf_size
)
Parameters:
devices [in] a set of devices handle. output_desc [in] refer to kp_generic_image_inference_result_header_t for describing some information of received data. raw_out_buffer [out] a user-allocated buffer for receiving the RAW data results, the needed buffer size can be known from the 'max_raw_out_size' in 'model_desc' through kp_load_model(). buf_size [in] size of raw_out_buffer.
Returns:
int refer to KP_API_RETURN_CODE in kp_struct.h
kp_generic_image_inference_send
Generic raw inference with multiple input images send.
int kp_generic_image_inference_send(
kp_device_group_t devices
kp_generic_image_inference_desc_t *inf_data
)
This is to perform one model inference with multiple input images, it is non-blocking if device buffer queue is not full.
When this is performed, user can issue kp_generic_image_inference_receive() to get the result.
In addition, to have better performance, users can issue multiple kp_generic_image_inference_send() then start to receive results through kp_generic_image_inference_receive().
Parameters:
devices [in] a set of devices handle. inf_data [in] inference data of needed parameters for performing inference including image buffer size, model id.
Returns:
int refer to KP_API_RETURN_CODE in kp_struct.h
kp_generic_inference_retrieve_fixed_node
Retrieve single node output data from raw output buffer.
kp_inf_fixed_node_output_t *kp_generic_inference_retrieve_fixed_node(
uint32_t node_idx
uint8_t *raw_out_buffer
kp_channel_ordering_t ordering
)
This function retrieves and converts RAW format data to fixed-point data on the per-node basis.
Parameters:
node_idx [in] wanted output node index, starts from 0. Number of total output nodes can be known from 'kp_generic_raw_result_header_t' raw_out_buffer [in] the RAW output buffer, it should come from kp_generic_raw_inference_receive(). ordering [in] the RAW output channel ordering
Returns:
refer to kp_inf_fixed_node_output_t. It describes fixed-point values of this node in specific channel ordering.
kp_generic_inference_retrieve_float_node
Retrieve single node output data from raw output buffer.
kp_inf_float_node_output_t *kp_generic_inference_retrieve_float_node(
uint32_t node_idx
uint8_t *raw_out_buffer
kp_channel_ordering_t ordering
)
This function retrieves and converts RAW format data to floating-point data on the per-node basis.
Parameters:
node_idx [in] wanted output node index, starts from 0. Number of total output nodes can be known from 'kp_generic_raw_result_header_t' raw_out_buffer [in] the RAW output buffer, it should come from kp_generic_raw_inference_receive(). ordering [in] the RAW output channel ordering
Returns:
refer to kp_inf_float_node_output_t. It describes floating-point values of this node in specific channel ordering.
kp_generic_inference_retrieve_raw_fixed_node
Retrieve single node output data from raw output buffer.
kp_inf_raw_fixed_node_output_t *kp_generic_inference_retrieve_raw_fixed_node(
uint32_t node_idx
uint8_t *raw_out_buffer
)
This function retrieves RAW format data in fixed-point format on the per-node basis.
The return pointer of 'kp_inf_raw_fixed_node_output_t' actually points to raw_out_buffer so do not free raw_out_buffer before completing the use of 'kp_inf_raw_fixed_node_output_t *'
Parameters:
node_idx [in] wanted output node index, starts from 0. Number of total output nodes can be known from 'kp_generic_raw_result_header_t' raw_out_buffer [in] the RAW output buffer, it should come from kp_generic_raw_inference_receive().
Returns:
refer to kp_inf_raw_fixed_node_output_t. It describes fixed-point values of this node with the Kneron device origin raw data buffer and channel ordering (KL520: height x channel x width (aligned to 16 byte), KL720: channel x height x width (aligned to 16 byte)).
kp_inference_configure
Configure inference settings.
int kp_inference_configure(
kp_device_group_t devices
kp_inf_configuration_t *conf
)
Parameters:
conf [in] refer to kp_inf_configuration_t.
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_profile_get_statistics
Collect inference profile results.
int kp_profile_get_statistics(
kp_device_group_t devices
kp_profile_data_t *profile_data
)
Parameters:
devices [in] a set of devices handle. profile_data [out] refer to kp_profile_data_t.
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_profile_set_enable
To set enable/disable debug profile.
int kp_profile_set_enable(
kp_device_group_t devices
bool enable
)
Parameters:
devices [in] a set of devices handle. enable [in] set enable/disable.
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_release_dbg_checkpoint_data
Release the debug checkpoint data
int kp_release_dbg_checkpoint_data(
void *checkpoint_buf
)
Parameters:
checkpoint_buf checkpoint buffer to be released
Returns:
refer to KP_API_RETURN_CODE in kp_struct.h
kp_release_fixed_node_output
Release the single node output data from raw output buffer
void kp_release_fixed_node_output(
kp_inf_fixed_node_output_t *fixed_node_output
)
Parameters:
fixed_node_output kp_inf_fixed_node_output_t to be released
kp_release_float_node_output
Release the single node output data from raw output buffer
void kp_release_float_node_output(
kp_inf_float_node_output_t *float_node_output
)
Parameters:
fixed_node_output kp_inf_float_node_output_t to be released
kp_release_raw_fixed_node_output
Release the single node output data from raw output buffer
void kp_release_raw_fixed_node_output(
kp_inf_raw_fixed_node_output_t *raw_fixed_node_output
)
Parameters:
raw_fixed_node_output kp_inf_raw_fixed_node_output_t to be released