Package org.freedesktop.gstreamer.video

Class VideoDecoder

java.lang.Object

io.github.jwharm.javagi.base.ProxyInstance

org.gnome.gobject.TypeInstance

org.gnome.gobject.GObject

org.gnome.gobject.InitiallyUnowned

org.freedesktop.gstreamer.gst.GstObject

org.freedesktop.gstreamer.gst.Element

org.freedesktop.gstreamer.video.VideoDecoder

All Implemented Interfaces:

Proxy

Direct Known Subclasses:

VideoDecoder.VideoDecoderImpl

@Generated("io.github.jwharm.JavaGI")
public abstract class VideoDecoder
extends Element

This base class is for video decoders turning encoded data into raw video frames.

The GstVideoDecoder base class and derived subclasses should cooperate as follows:

Configuration

• Initially, GstVideoDecoder calls start when the decoder element is activated, which allows the subclass to perform any global setup.

• GstVideoDecoder calls setFormat to inform the subclass of caps describing input video data that it is about to receive, including possibly configuration data. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.

• Incoming data buffers are processed as needed, described in Data Processing below.

• GstVideoDecoder calls stop at end of all processing.

Data processing

• The base class gathers input data, and optionally allows subclass to parse this into subsequently manageable chunks, typically corresponding to and referred to as 'frames'.

• Each input frame is provided in turn to the subclass' handleFrame callback.
• When the subclass enables the subframe mode with gst_video_decoder_set_subframe_mode, the base class will provide to the subclass the same input frame with different input buffers to the subclass handleFrame callback. During this call, the subclass needs to take ownership of the input_buffer as GstVideoCodecFrame.input_buffer will have been changed before the next subframe buffer is received. The subclass will call gst_video_decoder_have_last_subframe when a new input frame can be created by the base class. Every subframe will share the same GstVideoCodecFrame.output_buffer to write the decoding result. The subclass is responsible to protect its access.

• If codec processing results in decoded data, the subclass should call gstVideoDecoderFinishFrame to have decoded data pushed downstream. In subframe mode the subclass should call gstVideoDecoderFinishSubframe until the last subframe where it should call gstVideoDecoderFinishFrame. The subclass can detect the last subframe using GST_VIDEO_BUFFER_FLAG_MARKER on buffers or using its own logic to collect the subframes. In case of decoding failure, the subclass must call gstVideoDecoderDropFrame or gstVideoDecoderDropSubframe, to allow the base class to do timestamp and offset tracking, and possibly to requeue the frame for a later attempt in the case of reverse playback.

Shutdown phase

• The GstVideoDecoder class calls stop to inform the subclass that data parsing will be stopped.

Additional Notes

• Seeking/Flushing

• When the pipeline is seeked or otherwise flushed, the subclass is informed via a call to its reset callback, with the hard parameter set to true. This indicates the subclass should drop any internal data queues and timestamps and prepare for a fresh set of buffers to arrive for parsing and decoding.

• End Of Stream

• At end-of-stream, the subclass parse function may be called some final times with the at_eos parameter set to true, indicating that the element should not expect any more data to be arriving, and it should parse and remaining frames and call gst_video_decoder_have_frame() if possible.

The subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to provide information about the output caps, when they are known. This may be when the base class calls the subclass' setFormat function, though it might be during decoding, before calling gstVideoDecoderFinishFrame. This is done via gstVideoDecoderSetOutputState

The subclass is also responsible for providing (presentation) timestamps (likely based on corresponding input ones). If that is not applicable or possible, the base class provides limited framerate based interpolation.

Similarly, the base class provides some limited (legacy) seeking support if specifically requested by the subclass, as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates. To enable it, a subclass should call gstVideoDecoderSetEstimateRate to enable handling of incoming byte-streams.

The base class provides some support for reverse playback, in particular in case incoming data is not packetized or upstream does not provide fragments on keyframe boundaries. However, the subclass should then be prepared for the parsing and frame processing stage to occur separately (in normal forward processing, the latter immediately follows the former), The subclass also needs to ensure the parsing stage properly marks keyframes, unless it knows the upstream elements will do so properly for incoming data.

The bare minimum that a functional subclass needs to implement is:

• Provide pad templates
• Inform the base class of output caps via gstVideoDecoderSetOutputState

• Parse input data, if it is not considered packetized from upstream Data will be provided to parse which should invoke gstVideoDecoderAddToFrame and gstVideoDecoderHaveFrame to separate the data belonging to each video frame.

• Accept data in handleFrame and provide decoded results to gstVideoDecoderFinishFrame, or call gstVideoDecoderDropFrame.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	VideoDecoder.Builder<B extends VideoDecoder.Builder<B>>	Inner class implementing a builder pattern to construct a GObject with properties.
static class	VideoDecoder.VideoDecoderClass	Subclasses can override any of the available virtual methods or not, as needed.
static class	VideoDecoder.VideoDecoderImpl	The VideoDecoderImpl type represents a native instance of the abstract VideoDecoder class.

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.Element

Element.ElementClass, Element.ElementImpl, Element.NoMorePadsCallback, Element.PadAddedCallback, Element.PadRemovedCallback

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.GstObject

GstObject.DeepNotifyCallback, GstObject.ObjectClass, GstObject.ObjectImpl

Nested classes/interfaces inherited from class org.gnome.gobject.InitiallyUnowned

InitiallyUnowned.InitiallyUnownedClass

Nested classes/interfaces inherited from class org.gnome.gobject.GObject

GObject.NotifyCallback

Constructor Summary

Constructors

Constructor	Description
VideoDecoder(MemorySegment address)	Create a VideoDecoder proxy instance for the provided memory address.

Method Summary

Modifier and Type	Method	Description
void	addToFrame(int nBytes)	Removes next nBytes of input data and adds it to currently parsed frame.
Buffer	allocateOutputBuffer()	Helper function that allocates a buffer to hold a video frame for this VideoDecoder's current GstVideoCodecState.
FlowReturn	allocateOutputFrame(VideoCodecFrame frame)	Helper function that allocates a buffer to hold a video frame for this VideoDecoder's current GstVideoCodecState.
FlowReturn	allocateOutputFrameWithParams(VideoCodecFrame frame, BufferPoolAcquireParams params)	Same as gst_video_decoder_allocate_output_frame except it allows passing GstBufferPoolAcquireParams to the sub call gst_buffer_pool_acquire_buffer.
protected VideoDecoder	asParent()	Returns this instance as if it were its parent type.
static VideoDecoder.Builder<? extends VideoDecoder.Builder>	builder()	A VideoDecoder.Builder object constructs a VideoDecoder with the specified properties.
protected boolean	close()	Optional.
protected boolean	decideAllocation(Query query)	Optional.
protected FlowReturn	drain()	Optional.
FlowReturn	dropFrame(VideoCodecFrame frame)	Similar to gst_video_decoder_finish_frame(), but drops frame in any case and posts a QoS message with the frame's details on the bus.
FlowReturn	dropSubframe(VideoCodecFrame frame)	Drops input data.
protected FlowReturn	finish()	Optional.
FlowReturn	finishFrame(VideoCodecFrame frame)	frame should have a valid decoded data buffer, whose metadata fields are then appropriately set according to frame data and pushed downstream.
FlowReturn	finishSubframe(VideoCodecFrame frame)	Indicate that a subframe has been finished to be decoded by the subclass.
protected boolean	flush()	Optional.
void	getAllocator(@Nullable Out<Allocator> allocator, @Nullable AllocationParams params)	Lets GstVideoDecoder sub-classes to know the memory allocator used by the base class and its params.
BufferPool	getBufferPool()
protected Caps	getcaps(Caps filter)	Optional.
int	getEstimateRate()
VideoCodecFrame	getFrame(int frameNumber)	Get a pending unfinished GstVideoCodecFrame
List<VideoCodecFrame>	getFrames()	Get all pending unfinished GstVideoCodecFrame
int	getInputSubframeIndex(VideoCodecFrame frame)	Queries the number of the last subframe received by the decoder baseclass in the frame.
void	getLatency(@Nullable ClockTime minLatency, @Nullable ClockTime maxLatency)	Query the configured decoder latency.
ClockTimeDiff	getMaxDecodeTime(VideoCodecFrame frame)	Determines maximum possible decoding time for frame that will allow it to decode and arrive in time (as determined by QoS events).
int	getMaxErrors()
static MemoryLayout	getMemoryLayout()	The memory layout of the native struct.
boolean	getNeedsFormat()	Queries decoder required format handling.
boolean	getNeedsSyncPoint()	Queries if the decoder requires a sync point before it starts outputting data in the beginning.
VideoCodecFrame	getOldestFrame()	Get the oldest pending unfinished GstVideoCodecFrame
VideoCodecState	getOutputState()	Get the GstVideoCodecState currently describing the output stream.
boolean	getPacketized()	Queries whether input data is considered packetized or not by the base class.
long	getPendingFrameSize()	Returns the number of bytes previously added to the current frame by calling gst_video_decoder_add_to_frame().
int	getProcessedSubframeIndex(VideoCodecFrame frame)	Queries the number of subframes in the frame processed by the decoder baseclass.
double	getQosProportion()
boolean	getSubframeMode()	Queries whether input data is considered as subframes or not by the base class.
static Type	getType()	Get the GType of the VideoDecoder class
protected FlowReturn	handleFrame(VideoCodecFrame frame)
protected boolean	handleMissingData(ClockTime timestamp, ClockTime duration)
FlowReturn	haveFrame()	Gathers all data collected for currently parsed frame, gathers corresponding metadata and passes it along for further processing, i.e.
FlowReturn	haveLastSubframe(VideoCodecFrame frame)	Indicates that the last subframe has been processed by the decoder in frame.
void	mergeTags(@Nullable TagList tags, TagMergeMode mode)	Sets the audio decoder tags and how they should be merged with any upstream stream tags.
boolean	negotiate()	Negotiate with downstream elements to currently configured GstVideoCodecState.
protected boolean	open()	Optional.
protected FlowReturn	parse(VideoCodecFrame frame, Adapter adapter, boolean atEos)	Required for non-packetized input.
protected boolean	proposeAllocation(Query query)	Optional.
Caps	proxyGetcaps(@Nullable Caps caps, @Nullable Caps filter)	Returns caps that express caps (or sink template caps if caps == NULL) restricted to resolution/format/...
void	releaseFrame(VideoCodecFrame frame)	Similar to gst_video_decoder_drop_frame(), but simply releases frame without any processing other than removing it from list of pending frames, after which it is considered finished and released.
void	requestSyncPoint(VideoCodecFrame frame, Set<VideoDecoderRequestSyncPointFlags> flags)	Allows the GstVideoDecoder subclass to request from the base class that a new sync should be requested from upstream, and that frame was the frame when the subclass noticed that a new sync point is required.
void	requestSyncPoint(VideoCodecFrame frame, VideoDecoderRequestSyncPointFlags... flags)	Allows the GstVideoDecoder subclass to request from the base class that a new sync should be requested from upstream, and that frame was the frame when the subclass noticed that a new sync point is required.
protected boolean	reset(boolean hard)	Optional.
void	setEstimateRate(boolean enabled)	Allows baseclass to perform byte to time estimated conversion.
protected boolean	setFormat(VideoCodecState state)	Notifies subclass of incoming data format (caps).
VideoCodecState	setInterlacedOutputState(VideoFormat fmt, VideoInterlaceMode interlaceMode, int width, int height, @Nullable VideoCodecState reference)	Same as gst_video_decoder_set_output_state() but also allows you to also set the interlacing mode.
void	setLatency(ClockTime minLatency, ClockTime maxLatency)	Lets GstVideoDecoder sub-classes tell the baseclass what the decoder latency is.
void	setMaxErrors(int num)	Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error.
void	setNeedsFormat(boolean enabled)	Configures decoder format needs.
void	setNeedsSyncPoint(boolean enabled)	Configures whether the decoder requires a sync point before it starts outputting data in the beginning.
VideoCodecState	setOutputState(VideoFormat fmt, int width, int height, @Nullable VideoCodecState reference)	Creates a new GstVideoCodecState with the specified fmt, width and height as the output state for the decoder.
void	setPacketized(boolean packetized)	Allows baseclass to consider input data as packetized or not.
void	setSubframeMode(boolean subframeMode)	If this is set to TRUE, it informs the base class that the subclass can receive the data at a granularity lower than one frame.
void	setUseDefaultPadAcceptcaps(boolean use)	Lets GstVideoDecoder sub-classes decide if they want the sink pad to use the default pad query handler to reply to accept-caps queries.
protected boolean	sinkEvent(Event event)	Optional.
protected boolean	sinkQuery(Query query)	Optional.
protected boolean	srcEvent(Event event)	Optional.
protected boolean	srcQuery(Query query)	Optional.
protected boolean	start()	Optional.
protected boolean	stop()	Optional.
protected boolean	transformMeta(VideoCodecFrame frame, Meta meta)	Optional.

Methods inherited from class org.freedesktop.gstreamer.gst.Element

abortState, addPad, addPropertyDeepNotifyWatch, addPropertyNotifyWatch, callAsync, changeState, continueState, createAllPads, decorateStreamId, decorateStreamIdPrintf, emitNoMorePads, emitPadAdded, emitPadRemoved, foreachPad, foreachSinkPad, foreachSrcPad, getBaseTime, getBus, getClock, getCompatiblePad, getCompatiblePadTemplate, getContext, getContexts, getContextUnlocked, getCurrentClockTime, getCurrentRunningTime, getFactory, getMetadata, getPadTemplate, getPadTemplateList, getRequestPad, getStartTime, getState, getStaticPad, isLockedState, iteratePads, iterateSinkPads, iterateSrcPads, link, linkFiltered, linkMany, linkPads, linkPadsFiltered, linkPadsFull, linkPadsFull, lostState, makeFromUri, messageFull, messageFull, messageFullWithDetails, messageFullWithDetails, noMorePads, onNoMorePads, onPadAdded, onPadRemoved, padAdded, padRemoved, postMessage, provideClock, query, queryConvert, queryDuration, queryPosition, register, releasePad, releaseRequestPad, removePad, removePropertyNotifyWatch, requestPad, requestPadSimple, seek, seek, seekSimple, seekSimple, sendEvent, setBaseTime, setBus, setClock, setContext, setLockedState, setStartTime, setState, stateChanged, stateChangeReturnGetName, stateGetName, syncStateWithParent, typeSetSkipDocumentation, unlink, unlinkMany, unlinkPads

Methods inherited from class org.freedesktop.gstreamer.gst.GstObject

addControlBinding, checkUniqueness, deepNotify, defaultDeepNotify, defaultError, emitDeepNotify, getControlBinding, getControlRate, getGValueArray, getName, getParent, getPathString, getValue, getValueArray, hasActiveControlBindings, hasAncestor, hasAsAncestor, hasAsParent, onDeepNotify, ref, refSink, removeControlBinding, replace, setControlBindingDisabled, setControlBindingsDisabled, setControlRate, setName, setParent, suggestNextSync, syncValues, unparent, unref

Methods inherited from class org.gnome.gobject.GObject

addToggleRef, addWeakPointer, bindProperty, bindProperty, bindProperty, bindPropertyFull, bindPropertyFull, bindPropertyWithClosures, bindPropertyWithClosures, compatControl, connect, connect, connect, constructed, disconnect, dispatchPropertiesChanged, dispose, dupData, dupQdata, emit, emitNotify, finalize_, forceFloating, freezeNotify, get, getData, getProperty, getProperty, getProperty, getQdata, getv, interfaceFindProperty, interfaceInstallProperty, interfaceListProperties, isFloating, newInstance, newInstance, newInstance, newInstance, newv, notify_, notify_, notifyByPspec, onNotify, refSink, removeToggleRef, removeWeakPointer, replaceData, replaceQdata, runDispose, set, setData, setDataFull, setProperty, setProperty, setProperty, setQdata, setQdataFull, setv, stealData, stealQdata, takeRef, thawNotify, watchClosure, weakRef, weakUnref, withProperties

Methods inherited from class org.gnome.gobject.TypeInstance

callParent, callParent, getPrivate, readGClass, writeGClass

Methods inherited from class io.github.jwharm.javagi.base.ProxyInstance

equals, handle, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Details

VideoDecoder

public VideoDecoder(MemorySegment address)

Create a VideoDecoder proxy instance for the provided memory address.

Parameters:

address - the memory address of the native object

Method Details

getType

public static Type getType()

Get the GType of the VideoDecoder class

Returns:

the GType

getMemoryLayout

public static MemoryLayout getMemoryLayout()

The memory layout of the native struct.

Returns:

the memory layout

asParent

protected VideoDecoder asParent()

Returns this instance as if it were its parent type. This is mostly synonymous to the Java super keyword, but will set the native typeclass function pointers to the parent type. When overriding a native virtual method in Java, "chaining up" with super.methodName() doesn't work, because it invokes the overridden function pointer again. To chain up, call asParent().methodName(). This will call the native function pointer of this virtual method in the typeclass of the parent type.

Overrides:

asParent in class Element

addToFrame

public void addToFrame(int nBytes)

Removes next nBytes of input data and adds it to currently parsed frame.

Parameters:

nBytes - the number of bytes to add

allocateOutputBuffer

public Buffer allocateOutputBuffer()

Helper function that allocates a buffer to hold a video frame for this VideoDecoder's current GstVideoCodecState.

You should use gst_video_decoder_allocate_output_frame() instead of this function, if possible at all.

Returns:

allocated buffer, or NULL if no buffer could be allocated (e.g. when downstream is flushing or shutting down)

allocateOutputFrame

public FlowReturn allocateOutputFrame(VideoCodecFrame frame)

Helper function that allocates a buffer to hold a video frame for this VideoDecoder's current GstVideoCodecState. Subclass should already have configured video state and set src pad caps.

The buffer allocated here is owned by the frame and you should only keep references to the frame, not the buffer.

Parameters:

frame - a GstVideoCodecFrame

Returns:

FlowReturn.OK if an output buffer could be allocated

allocateOutputFrameWithParams

public FlowReturn allocateOutputFrameWithParams(VideoCodecFrame frame,
 BufferPoolAcquireParams params)

Same as gst_video_decoder_allocate_output_frame except it allows passing GstBufferPoolAcquireParams to the sub call gst_buffer_pool_acquire_buffer.

Parameters:

frame - a GstVideoCodecFrame

params - a GstBufferPoolAcquireParams

Returns:

FlowReturn.OK if an output buffer could be allocated

dropFrame

public FlowReturn dropFrame(VideoCodecFrame frame)

Similar to gst_video_decoder_finish_frame(), but drops frame in any case and posts a QoS message with the frame's details on the bus. In any case, the frame is considered finished and released.

Parameters:

frame - the GstVideoCodecFrame to drop

Returns:

a GstFlowReturn, usually GST_FLOW_OK.

dropSubframe

public FlowReturn dropSubframe(VideoCodecFrame frame)

Drops input data. The frame is not considered finished until the whole frame is finished or dropped by the subclass.

Parameters:

frame - the GstVideoCodecFrame

Returns:

a GstFlowReturn, usually GST_FLOW_OK.

finishFrame

public FlowReturn finishFrame(VideoCodecFrame frame)

frame should have a valid decoded data buffer, whose metadata fields are then appropriately set according to frame data and pushed downstream. If no output data is provided, frame is considered skipped. In any case, the frame is considered finished and released.

After calling this function the output buffer of the frame is to be considered read-only. This function will also change the metadata of the buffer.

Parameters:

frame - a decoded GstVideoCodecFrame

Returns:

a GstFlowReturn resulting from sending data downstream

finishSubframe

public FlowReturn finishSubframe(VideoCodecFrame frame)

Indicate that a subframe has been finished to be decoded by the subclass. This method should be called for all subframes except the last subframe where gstVideoDecoderFinishFrame should be called instead.

Parameters:

frame - the GstVideoCodecFrame

Returns:

a GstFlowReturn, usually GST_FLOW_OK.

getAllocator

public void getAllocator(@Nullable
 @Nullable Out<Allocator> allocator,
 @Nullable
 @Nullable AllocationParams params)

Lets GstVideoDecoder sub-classes to know the memory allocator used by the base class and its params.

Unref the allocator after use it.

Parameters:

allocator - the GstAllocator used

params - the GstAllocationParams of allocator

getBufferPool

public BufferPool getBufferPool()

getEstimateRate

public int getEstimateRate()

getFrame

public VideoCodecFrame getFrame(int frameNumber)

Get a pending unfinished GstVideoCodecFrame

Parameters:

frameNumber - system_frame_number of a frame

Returns:

pending unfinished GstVideoCodecFrame identified by frameNumber.

getFrames

public List<VideoCodecFrame> getFrames()

Get all pending unfinished GstVideoCodecFrame

Returns:

pending unfinished GstVideoCodecFrame.

getInputSubframeIndex

public int getInputSubframeIndex(VideoCodecFrame frame)

Queries the number of the last subframe received by the decoder baseclass in the frame.

Parameters:

frame - the GstVideoCodecFrame to update

Returns:

the current subframe index received in subframe mode, 1 otherwise.

getLatency

public void getLatency(@Nullable
 @Nullable ClockTime minLatency,
 @Nullable
 @Nullable ClockTime maxLatency)

Query the configured decoder latency. Results will be returned via minLatency and maxLatency.

Parameters:

minLatency - address of variable in which to store the configured minimum latency, or null

maxLatency - address of variable in which to store the configured mximum latency, or null

getMaxDecodeTime

public ClockTimeDiff getMaxDecodeTime(VideoCodecFrame frame)

Determines maximum possible decoding time for frame that will allow it to decode and arrive in time (as determined by QoS events). In particular, a negative result means decoding in time is no longer possible and should therefore occur as soon/skippy as possible.

Parameters:

frame - a GstVideoCodecFrame

Returns:

max decoding time.

getMaxErrors

public int getMaxErrors()

getNeedsFormat

public boolean getNeedsFormat()

Queries decoder required format handling.

Returns:

true if required format handling is enabled.

getNeedsSyncPoint

public boolean getNeedsSyncPoint()

Queries if the decoder requires a sync point before it starts outputting data in the beginning.

Returns:

true if a sync point is required in the beginning.

getOldestFrame

public VideoCodecFrame getOldestFrame()

Get the oldest pending unfinished GstVideoCodecFrame

Returns:

oldest pending unfinished GstVideoCodecFrame.

getOutputState

public VideoCodecState getOutputState()

Get the GstVideoCodecState currently describing the output stream.

Returns:

GstVideoCodecState describing format of video data.

getPacketized

public boolean getPacketized()

Queries whether input data is considered packetized or not by the base class.

Returns:

TRUE if input data is considered packetized.

getPendingFrameSize

public long getPendingFrameSize()

Returns the number of bytes previously added to the current frame by calling gst_video_decoder_add_to_frame().

Returns:

The number of bytes pending for the current frame

getProcessedSubframeIndex

public int getProcessedSubframeIndex(VideoCodecFrame frame)

Queries the number of subframes in the frame processed by the decoder baseclass.

Parameters:

frame - the GstVideoCodecFrame to update

Returns:

the current subframe processed received in subframe mode.

getQosProportion

public double getQosProportion()

getSubframeMode

public boolean getSubframeMode()

Queries whether input data is considered as subframes or not by the base class. If FALSE, each input buffer will be considered as a full frame.

Returns:

TRUE if input data is considered as sub frames.

haveFrame

public FlowReturn haveFrame()

Gathers all data collected for currently parsed frame, gathers corresponding metadata and passes it along for further processing, i.e. handleFrame.

Returns:

a GstFlowReturn

haveLastSubframe

public FlowReturn haveLastSubframe(VideoCodecFrame frame)

Indicates that the last subframe has been processed by the decoder in frame. This will release the current frame in video decoder allowing to receive new frames from upstream elements. This method must be called in the subclass handleFrame callback.

Parameters:

frame - the GstVideoCodecFrame to update

Returns:

a GstFlowReturn, usually GST_FLOW_OK.

mergeTags

public void mergeTags(@Nullable
 @Nullable TagList tags,
 TagMergeMode mode)

Sets the audio decoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_audio_decoder_merge_tags().

Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.

MT safe.

Parameters:

tags - a GstTagList to merge, or NULL to unset previously-set tags

mode - the GstTagMergeMode to use, usually GST_TAG_MERGE_REPLACE

negotiate

public boolean negotiate()

Negotiate with downstream elements to currently configured GstVideoCodecState. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.

Returns:

true if the negotiation succeeded, else false.

proxyGetcaps

public Caps proxyGetcaps(@Nullable
 @Nullable Caps caps,
 @Nullable
 @Nullable Caps filter)

Returns caps that express caps (or sink template caps if caps == NULL) restricted to resolution/format/... combinations supported by downstream elements.

Parameters:

caps - initial caps

filter - filter caps

Returns:

a GstCaps owned by caller

releaseFrame

public void releaseFrame(VideoCodecFrame frame)

Similar to gst_video_decoder_drop_frame(), but simply releases frame without any processing other than removing it from list of pending frames, after which it is considered finished and released.

Parameters:

frame - the GstVideoCodecFrame to release

requestSyncPoint

public void requestSyncPoint(VideoCodecFrame frame,
 Set<VideoDecoderRequestSyncPointFlags> flags)

Allows the GstVideoDecoder subclass to request from the base class that a new sync should be requested from upstream, and that frame was the frame when the subclass noticed that a new sync point is required. A reason for the subclass to do this could be missing reference frames, for example.

The base class will then request a new sync point from upstream as long as the time that passed since the last one is exceeding GstVideoDecoder:min-force-key-unit-interval.

The subclass can signal via flags how the frames until the next sync point should be handled:

• If VideoDecoderRequestSyncPointFlags.DISCARD_INPUT is selected then all following input frames until the next sync point are discarded. This can be useful if the lack of a sync point will prevent all further decoding and the decoder implementation is not very robust in handling missing references frames.
• If VideoDecoderRequestSyncPointFlags.CORRUPT_OUTPUT is selected then all output frames following frame are marked as corrupted via BufferFlags.CORRUPTED. Corrupted frames can be automatically dropped by the base class, see GstVideoDecoder:discard-corrupted-frames. Subclasses can manually mark frames as corrupted via VideoCodecFrameFlags.CORRUPTED before calling gst_video_decoder_finish_frame().

Parameters:

frame - a GstVideoCodecFrame

flags - GstVideoDecoderRequestSyncPointFlags

requestSyncPoint

public void requestSyncPoint(VideoCodecFrame frame,
 VideoDecoderRequestSyncPointFlags... flags)

Allows the GstVideoDecoder subclass to request from the base class that a new sync should be requested from upstream, and that frame was the frame when the subclass noticed that a new sync point is required. A reason for the subclass to do this could be missing reference frames, for example.

The base class will then request a new sync point from upstream as long as the time that passed since the last one is exceeding GstVideoDecoder:min-force-key-unit-interval.

The subclass can signal via flags how the frames until the next sync point should be handled:

• If VideoDecoderRequestSyncPointFlags.DISCARD_INPUT is selected then all following input frames until the next sync point are discarded. This can be useful if the lack of a sync point will prevent all further decoding and the decoder implementation is not very robust in handling missing references frames.
• If VideoDecoderRequestSyncPointFlags.CORRUPT_OUTPUT is selected then all output frames following frame are marked as corrupted via BufferFlags.CORRUPTED. Corrupted frames can be automatically dropped by the base class, see GstVideoDecoder:discard-corrupted-frames. Subclasses can manually mark frames as corrupted via VideoCodecFrameFlags.CORRUPTED before calling gst_video_decoder_finish_frame().

Parameters:

frame - a GstVideoCodecFrame

flags - GstVideoDecoderRequestSyncPointFlags

setEstimateRate

public void setEstimateRate(boolean enabled)

Allows baseclass to perform byte to time estimated conversion.

Parameters:

enabled - whether to enable byte to time conversion

setInterlacedOutputState

public VideoCodecState setInterlacedOutputState(VideoFormat fmt,
 VideoInterlaceMode interlaceMode,
 int width,
 int height,
 @Nullable
 @Nullable VideoCodecState reference)

Same as gst_video_decoder_set_output_state() but also allows you to also set the interlacing mode.

Parameters:

fmt - a GstVideoFormat

interlaceMode - A GstVideoInterlaceMode

width - The width in pixels

height - The height in pixels

reference - An optional reference GstVideoCodecState

Returns:

the newly configured output state.

setLatency

public void setLatency(ClockTime minLatency,
 ClockTime maxLatency)

Lets GstVideoDecoder sub-classes tell the baseclass what the decoder latency is. If the provided values changed from previously provided ones, this will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.

Parameters:

minLatency - minimum latency

maxLatency - maximum latency

setMaxErrors

public void setMaxErrors(int num)

Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error. You can set -1 for never returning fatal errors. Default is set to GST_VIDEO_DECODER_MAX_ERRORS.

The '-1' option was added in 1.4

Parameters:

num - max tolerated errors

setNeedsFormat

public void setNeedsFormat(boolean enabled)

Configures decoder format needs. If enabled, subclass needs to be negotiated with format caps before it can process any data. It will then never be handed any data before it has been configured. Otherwise, it might be handed data without having been configured and is then expected being able to do so either by default or based on the input data.

Parameters:

enabled - new state

setNeedsSyncPoint

public void setNeedsSyncPoint(boolean enabled)

Configures whether the decoder requires a sync point before it starts outputting data in the beginning. If enabled, the base class will discard all non-sync point frames in the beginning and after a flush and does not pass it to the subclass.

If the first frame is not a sync point, the base class will request a sync point via the force-key-unit event.

Parameters:

enabled - new state

setOutputState

public VideoCodecState setOutputState(VideoFormat fmt,
 int width,
 int height,
 @Nullable
 @Nullable VideoCodecState reference)

Creates a new GstVideoCodecState with the specified fmt, width and height as the output state for the decoder. Any previously set output state on this VideoDecoder will be replaced by the newly created one.

If the subclass wishes to copy over existing fields (like pixel aspec ratio, or framerate) from an existing GstVideoCodecState, it can be provided as a reference.

If the subclass wishes to override some fields from the output state (like pixel-aspect-ratio or framerate) it can do so on the returned GstVideoCodecState.

The new output state will only take effect (set on pads and buffers) starting from the next call to gst_video_decoder_finish_frame().

Parameters:

fmt - a GstVideoFormat

width - The width in pixels

height - The height in pixels

reference - An optional reference GstVideoCodecState

Returns:

the newly configured output state.

setPacketized

public void setPacketized(boolean packetized)

Allows baseclass to consider input data as packetized or not. If the input is packetized, then the parse method will not be called.

Parameters:

packetized - whether the input data should be considered as packetized.

setSubframeMode

public void setSubframeMode(boolean subframeMode)

If this is set to TRUE, it informs the base class that the subclass can receive the data at a granularity lower than one frame.

Note that in this mode, the subclass has two options. It can either require the presence of a GST_VIDEO_BUFFER_FLAG_MARKER to mark the end of a frame. Or it can operate in such a way that it will decode a single frame at a time. In this second case, every buffer that arrives to the element is considered part of the same frame until gst_video_decoder_finish_frame() is called.

In either case, the same GstVideoCodecFrame will be passed to the GstVideoDecoderClass:handle_frame vmethod repeatedly with a different GstVideoCodecFrame:input_buffer every time until the end of the frame has been signaled using either method. This method must be called during the decoder subclass setFormat call.

Parameters:

subframeMode - whether the input data should be considered as subframes.

setUseDefaultPadAcceptcaps

public void setUseDefaultPadAcceptcaps(boolean use)

Lets GstVideoDecoder sub-classes decide if they want the sink pad to use the default pad query handler to reply to accept-caps queries.

By setting this to true it is possible to further customize the default handler with GST_PAD_SET_ACCEPT_INTERSECT and GST_PAD_SET_ACCEPT_TEMPLATE

Parameters:

use - if the default pad accept-caps query handling should be used

close

protected boolean close()

Optional. Called when the element changes to GST_STATE_NULL. Allows closing external resources.

decideAllocation

protected boolean decideAllocation(Query query)

Optional. Setup the allocation parameters for allocating output buffers. The passed in query contains the result of the downstream allocation query. Subclasses should chain up to the parent implementation to invoke the default handler.

drain

protected FlowReturn drain()

Optional. Called to request subclass to decode any data it can at this point, but that more data may arrive after. (e.g. at segment end). Sub-classes should be prepared to handle new data afterward, or seamless segment processing will break. Since: 1.6

finish

protected FlowReturn finish()

Optional. Called to request subclass to dispatch any pending remaining data at EOS. Sub-classes can refuse to decode new data after.

flush

protected boolean flush()

Optional. Flush all remaining data from the decoder without pushing it downstream. Since: 1.2

getcaps

protected Caps getcaps(Caps filter)

Optional. Allows for a custom sink getcaps implementation. If not implemented, default returns gst_video_decoder_proxy_getcaps applied to sink template caps.

handleFrame

protected FlowReturn handleFrame(VideoCodecFrame frame)

handleMissingData

protected boolean handleMissingData(ClockTime timestamp,
 ClockTime duration)

open

protected boolean open()

Optional. Called when the element changes to GST_STATE_READY. Allows opening external resources.

parse

protected FlowReturn parse(VideoCodecFrame frame,
 Adapter adapter,
 boolean atEos)

Required for non-packetized input. Allows chopping incoming data into manageable units (frames) for subsequent decoding.

proposeAllocation

protected boolean proposeAllocation(Query query)

Optional. Propose buffer allocation parameters for upstream elements. Subclasses should chain up to the parent implementation to invoke the default handler.

reset

protected boolean reset(boolean hard)

Optional. Allows subclass (decoder) to perform post-seek semantics reset. Deprecated.

setFormat

protected boolean setFormat(VideoCodecState state)

Notifies subclass of incoming data format (caps).

sinkEvent

protected boolean sinkEvent(Event event)

Optional. Event handler on the sink pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed). Subclasses should chain up to the parent implementation to invoke the default handler.

sinkQuery

protected boolean sinkQuery(Query query)

Optional. Query handler on the sink pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.4

srcEvent

protected boolean srcEvent(Event event)

Optional. Event handler on the source pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed). Subclasses should chain up to the parent implementation to invoke the default handler.

srcQuery

protected boolean srcQuery(Query query)

Optional. Query handler on the source pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.4

start

protected boolean start()

Optional. Called when the element starts processing. Allows opening external resources.

stop

protected boolean stop()

Optional. Called when the element stops processing. Allows closing external resources.

transformMeta

protected boolean transformMeta(VideoCodecFrame frame,
 Meta meta)

Optional. Transform the metadata on the input buffer to the output buffer. By default this method is copies all meta without tags and meta with only the "video" tag. subclasses can implement this method and return true if the metadata is to be copied. Since: 1.6

builder

public static VideoDecoder.Builder<? extends VideoDecoder.Builder> builder()

A VideoDecoder.Builder object constructs a VideoDecoder with the specified properties. Use the various set...() methods to set properties, and finish construction with VideoDecoder.Builder.build().