Package org.freedesktop.gstreamer.base

Class BaseParse

java.lang.Object

io.github.jwharm.javagi.base.ProxyInstance

org.gnome.gobject.TypeInstance

org.gnome.gobject.GObject

org.gnome.gobject.InitiallyUnowned

org.freedesktop.gstreamer.gst.GstObject

org.freedesktop.gstreamer.gst.Element

org.freedesktop.gstreamer.base.BaseParse

All Implemented Interfaces:

Proxy

Direct Known Subclasses:

BaseParse.BaseParseImpl

@Generated("io.github.jwharm.JavaGI")
public abstract class BaseParse
extends Element

This base class is for parser elements that process data and splits it into separate audio/video/whatever frames.

It provides for:

• provides one sink pad and one source pad
• handles state changes
• can operate in pull mode or push mode
• handles seeking in both modes
• handles events (SEGMENT/EOS/FLUSH)
• handles queries (POSITION/DURATION/SEEKING/FORMAT/CONVERT)
• handles flushing

The purpose of this base class is to provide the basic functionality of a parser and share a lot of rather complex code.

Description of the parsing mechanism:
Set-up phase

• GstBaseParse calls GstBaseParseClass::start to inform subclass that data processing is about to start now.

• GstBaseParse class calls GstBaseParseClass::set_sink_caps to inform the subclass about incoming sinkpad caps. Subclass could already set the srcpad caps accordingly, but this might be delayed until calling gst_base_parse_finish_frame() with a non-queued frame.

• At least at this point subclass needs to tell the GstBaseParse class how big data chunks it wants to receive (minimum frame size ). It can do this with gst_base_parse_set_min_frame_size().

• GstBaseParse class sets up appropriate data passing mode (pull/push) and starts to process the data.

Parsing phase

• GstBaseParse gathers at least min_frame_size bytes of data either by pulling it from upstream or collecting buffers in an internal GstAdapter.

• A buffer of (at least) min_frame_size bytes is passed to subclass with GstBaseParseClass::handle_frame. Subclass checks the contents and can optionally return GST_FLOW_OK along with an amount of data to be skipped to find a valid frame (which will result in a subsequent DISCONT). If, otherwise, the buffer does not hold a complete frame, GstBaseParseClass::handle_frame can merely return and will be called again when additional data is available. In push mode this amounts to an additional input buffer (thus minimal additional latency), in pull mode this amounts to some arbitrary reasonable buffer size increase.

Of course, gst_base_parse_set_min_frame_size() could also be used if a very specific known amount of additional data is required. If, however, the buffer holds a complete valid frame, it can pass the size of this frame to gst_base_parse_finish_frame().

If acting as a converter, it can also merely indicate consumed input data while simultaneously providing custom output data. Note that baseclass performs some processing (such as tracking overall consumed data rate versus duration) for each finished frame, but other state is only updated upon each call to GstBaseParseClass::handle_frame (such as tracking upstream input timestamp).

Subclass is also responsible for setting the buffer metadata (e.g. buffer timestamp and duration, or keyframe if applicable). (although the latter can also be done by GstBaseParse if it is appropriately configured, see below). Frame is provided with timestamp derived from upstream (as much as generally possible), duration obtained from configuration (see below), and offset if meaningful (in pull mode).

Note that GstBaseParseClass::handle_frame might receive any small amount of input data when leftover data is being drained (e.g. at EOS).

• As part of finish frame processing, just prior to actually pushing the buffer in question, it is passed to GstBaseParseClass::pre_push_frame which gives subclass yet one last chance to examine buffer metadata, or to send some custom (tag) events, or to perform custom (segment) filtering.

• During the parsing process GstBaseParseClass will handle both srcpad and sinkpad events. They will be passed to subclass if GstBaseParseClass::sink_event or GstBaseParseClass::src_event implementations have been provided.

Shutdown phase

• GstBaseParse class calls GstBaseParseClass::stop to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured (e.g. when base class calls subclass' GstBaseParseClass::set_sink_caps function).

This base class uses Format.DEFAULT as a meaning of frames. So, subclass conversion routine needs to know that conversion from Format.TIME to Format.DEFAULT must return the frame number that can be found from the given byte position.

GstBaseParse uses subclasses conversion methods also for seeking (or otherwise uses its own default one, see also below).

Subclass start and stop functions will be called to inform the beginning and end of data processing.

Things that subclass need to take care of:

• Provide pad templates
• Fixate the source pad caps when appropriate
• Inform base class how big data chunks should be retrieved. This is done with gst_base_parse_set_min_frame_size() function.
• Examine data chunks passed to subclass with GstBaseParseClass::handle_frame and pass proper frame(s) to gst_base_parse_finish_frame(), and setting src pad caps and timestamps on frame.
• Provide conversion functions
• Update the duration information with gst_base_parse_set_duration()
• Optionally passthrough using gst_base_parse_set_passthrough()
• Configure various baseparse parameters using gst_base_parse_set_average_bitrate(), gst_base_parse_set_syncable() and gst_base_parse_set_frame_rate().

• In particular, if subclass is unable to determine a duration, but parsing (or specs) yields a frames per seconds rate, then this can be provided to GstBaseParse to enable it to cater for buffer time metadata (which will be taken from upstream as much as possible). Internally keeping track of frame durations and respective sizes that have been pushed provides GstBaseParse with an estimated bitrate. A default GstBaseParseClass::convert (used if not overridden) will then use these rates to perform obvious conversions. These rates are also used to update (estimated) duration at regular frame intervals.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	BaseParse.BaseParseClass	Subclasses can override any of the available virtual methods or not, as needed.
static class	BaseParse.BaseParseImpl	The BaseParseImpl type represents a native instance of the abstract BaseParse class.
static class	BaseParse.Builder<B extends BaseParse.Builder<B>>	Inner class implementing a builder pattern to construct a GObject with properties.

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.Element

Element.ElementClass, Element.ElementImpl, Element.NoMorePadsCallback, Element.PadAddedCallback, Element.PadRemovedCallback

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.GstObject

GstObject.DeepNotifyCallback, GstObject.ObjectClass, GstObject.ObjectImpl

Nested classes/interfaces inherited from class org.gnome.gobject.InitiallyUnowned

InitiallyUnowned.InitiallyUnownedClass

Nested classes/interfaces inherited from class org.gnome.gobject.GObject

GObject.NotifyCallback

Constructor Summary

Constructors

Constructor	Description
BaseParse(MemorySegment address)	Create a BaseParse proxy instance for the provided memory address.

Method Summary

Modifier and Type	Method	Description
boolean	addIndexEntry(long offset, ClockTime ts, boolean key, boolean force)	Adds an entry to the index associating offset to ts.
protected BaseParse	asParent()	Returns this instance as if it were its parent type.
static BaseParse.Builder<? extends BaseParse.Builder>	builder()	A BaseParse.Builder object constructs a BaseParse with the specified properties.
protected boolean	convert(Format srcFormat, long srcValue, Format destFormat, MemorySegment destValue)	Optional.
boolean	convertDefault(Format srcFormat, long srcValue, Format destFormat, Out<Long> destValue)	Default implementation of GstBaseParseClass::convert.
protected FlowReturn	detect(Buffer buffer)	Optional.
void	drain()	Drains the adapter until it is empty.
FlowReturn	finishFrame(BaseParseFrame frame, int size)	Collects parsed data and pushes it downstream.
static MemoryLayout	getMemoryLayout()	The memory layout of the native struct.
protected Caps	getSinkCaps(Caps filter)	Optional.
static Type	getType()	Get the GType of the BaseParse class
protected FlowReturn	handleFrame(BaseParseFrame frame, Out<Integer> skipsize)	Parses the input data into valid frames as defined by subclass which should be passed to gst_base_parse_finish_frame().
void	mergeTags(@Nullable TagList tags, TagMergeMode mode)	Sets the parser subclass's tags and how they should be merged with any upstream stream tags.
protected FlowReturn	prePushFrame(BaseParseFrame frame)	Optional.
FlowReturn	pushFrame(BaseParseFrame frame)	Pushes the frame's buffer downstream, sends any pending events and does some timestamp and segment handling.
void	setAverageBitrate(int bitrate)	Optionally sets the average bitrate detected in media (if non-zero), e.g.
void	setDuration(Format fmt, long duration, int interval)	Sets the duration of the currently playing media.
void	setFrameRate(int fpsNum, int fpsDen, int leadIn, int leadOut)	If frames per second is configured, parser can take care of buffer duration and timestamping.
void	setHasTimingInfo(boolean hasTiming)	Set if frames carry timing information which the subclass can (generally) parse and provide.
void	setInferTs(boolean inferTs)	By default, the base class might try to infer PTS from DTS and vice versa.
void	setLatency(ClockTime minLatency, ClockTime maxLatency)	Sets the minimum and maximum (which may likely be equal) latency introduced by the parsing process.
void	setMinFrameSize(int minSize)	Subclass can use this function to tell the base class that it needs to be given buffers of at least minSize bytes.
void	setPassthrough(boolean passthrough)	Set if the nature of the format or configuration does not allow (much) parsing, and the parser should operate in passthrough mode (which only applies when operating in push mode).
void	setPtsInterpolation(boolean ptsInterpolate)	By default, the base class will guess PTS timestamps using a simple interpolation (previous timestamp + duration), which is incorrect for data streams with reordering, where PTS can go backward.
protected boolean	setSinkCaps(Caps caps)	Optional.
void	setSyncable(boolean syncable)	Set if frame starts can be identified.
void	setTsAtOffset(long offset)	This function should only be called from a handleFrame implementation.
protected boolean	sinkEvent(Event event)	Optional.
protected boolean	sinkQuery(Query query)	Optional.
protected boolean	srcEvent(Event event)	Optional.
protected boolean	srcQuery(Query query)	Optional.
protected boolean	start()	Optional.
protected boolean	stop()	Optional.

Methods inherited from class org.freedesktop.gstreamer.gst.Element

abortState, addPad, addPropertyDeepNotifyWatch, addPropertyNotifyWatch, callAsync, changeState, continueState, createAllPads, decorateStreamId, decorateStreamIdPrintf, emitNoMorePads, emitPadAdded, emitPadRemoved, foreachPad, foreachSinkPad, foreachSrcPad, getBaseTime, getBus, getClock, getCompatiblePad, getCompatiblePadTemplate, getContext, getContexts, getContextUnlocked, getCurrentClockTime, getCurrentRunningTime, getFactory, getMetadata, getPadTemplate, getPadTemplateList, getRequestPad, getStartTime, getState, getStaticPad, isLockedState, iteratePads, iterateSinkPads, iterateSrcPads, link, linkFiltered, linkMany, linkPads, linkPadsFiltered, linkPadsFull, linkPadsFull, lostState, makeFromUri, messageFull, messageFull, messageFullWithDetails, messageFullWithDetails, noMorePads, onNoMorePads, onPadAdded, onPadRemoved, padAdded, padRemoved, postMessage, provideClock, query, queryConvert, queryDuration, queryPosition, register, releasePad, releaseRequestPad, removePad, removePropertyNotifyWatch, requestPad, requestPadSimple, seek, seek, seekSimple, seekSimple, sendEvent, setBaseTime, setBus, setClock, setContext, setLockedState, setStartTime, setState, stateChanged, stateChangeReturnGetName, stateGetName, syncStateWithParent, typeSetSkipDocumentation, unlink, unlinkMany, unlinkPads

Methods inherited from class org.freedesktop.gstreamer.gst.GstObject

addControlBinding, checkUniqueness, deepNotify, defaultDeepNotify, defaultError, emitDeepNotify, getControlBinding, getControlRate, getGValueArray, getName, getParent, getPathString, getValue, getValueArray, hasActiveControlBindings, hasAncestor, hasAsAncestor, hasAsParent, onDeepNotify, ref, refSink, removeControlBinding, replace, setControlBindingDisabled, setControlBindingsDisabled, setControlRate, setName, setParent, suggestNextSync, syncValues, unparent, unref

Methods inherited from class org.gnome.gobject.GObject

addToggleRef, addWeakPointer, bindProperty, bindProperty, bindProperty, bindPropertyFull, bindPropertyFull, bindPropertyWithClosures, bindPropertyWithClosures, compatControl, connect, connect, connect, constructed, disconnect, dispatchPropertiesChanged, dispose, dupData, dupQdata, emit, emitNotify, finalize_, forceFloating, freezeNotify, get, getData, getProperty, getProperty, getProperty, getQdata, getv, interfaceFindProperty, interfaceInstallProperty, interfaceListProperties, isFloating, newInstance, newInstance, newv, notify, notify, notifyByPspec, onNotify, refSink, removeToggleRef, removeWeakPointer, replaceData, replaceQdata, runDispose, set, setData, setDataFull, setProperty, setProperty, setProperty, setQdata, setQdataFull, setv, stealData, stealQdata, takeRef, thawNotify, watchClosure, weakRef, weakUnref, withProperties

Methods inherited from class org.gnome.gobject.TypeInstance

callParent, callParent, getPrivate, readGClass, writeGClass

Methods inherited from class io.github.jwharm.javagi.base.ProxyInstance

equals, handle, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Details

BaseParse

public BaseParse(MemorySegment address)

Create a BaseParse proxy instance for the provided memory address.

Parameters:

address - the memory address of the native object

Method Details

getType

public static Type getType()

Get the GType of the BaseParse class

Returns:

the GType

getMemoryLayout

public static MemoryLayout getMemoryLayout()

The memory layout of the native struct.

Returns:

the memory layout

asParent

protected BaseParse asParent()

Returns this instance as if it were its parent type. This is mostly synonymous to the Java super keyword, but will set the native typeclass function pointers to the parent type. When overriding a native virtual method in Java, "chaining up" with super.methodName() doesn't work, because it invokes the overridden function pointer again. To chain up, call asParent().methodName(). This will call the native function pointer of this virtual method in the typeclass of the parent type.

Overrides:

asParent in class Element

addIndexEntry

public boolean addIndexEntry(long offset,
 ClockTime ts,
 boolean key,
 boolean force)

Adds an entry to the index associating offset to ts. It is recommended to only add keyframe entries. force allows to bypass checks, such as whether the stream is (upstream) seekable, another entry is already "close" to the new entry, etc.

Parameters:

offset - offset of entry

ts - timestamp associated with offset

key - whether entry refers to keyframe

force - add entry disregarding sanity checks

Returns:

gboolean indicating whether entry was added

convertDefault

public boolean convertDefault(Format srcFormat,
 long srcValue,
 Format destFormat,
 Out<Long> destValue)

Default implementation of GstBaseParseClass::convert.

Parameters:

srcFormat - GstFormat describing the source format.

srcValue - Source value to be converted.

destFormat - GstFormat defining the converted format.

destValue - Pointer where the conversion result will be put.

Returns:

true if conversion was successful.

drain

public void drain()

Drains the adapter until it is empty. It decreases the min_frame_size to match the current adapter size and calls chain method until the adapter is emptied or chain returns with error.

finishFrame

public FlowReturn finishFrame(BaseParseFrame frame,
 int size)

Collects parsed data and pushes it downstream. Source pad caps must be set when this is called.

If frame's out_buffer is set, that will be used as subsequent frame data, and size amount will be flushed from the input data. The output_buffer size can differ from the consumed size indicated by size.

Otherwise, size samples will be taken from the input and used for output, and the output's metadata (timestamps etc) will be taken as (optionally) set by the subclass on frame's (input) buffer (which is otherwise ignored for any but the above purpose/information).

Note that the latter buffer is invalidated by this call, whereas the caller retains ownership of frame.

Parameters:

frame - a GstBaseParseFrame

size - consumed input data represented by frame

Returns:

a GstFlowReturn that should be escalated to caller (of caller)

mergeTags

public void mergeTags(@Nullable
 @Nullable TagList tags,
 TagMergeMode mode)

Sets the parser subclass's tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_base_parse_merge_tags().

Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.

Parameters:

tags - a GstTagList to merge, or NULL to unset previously-set tags

mode - the GstTagMergeMode to use, usually GST_TAG_MERGE_REPLACE

pushFrame

public FlowReturn pushFrame(BaseParseFrame frame)

Pushes the frame's buffer downstream, sends any pending events and does some timestamp and segment handling. Takes ownership of frame's buffer, though caller retains ownership of frame.

This must be called with sinkpad STREAM_LOCK held.

Parameters:

frame - a GstBaseParseFrame

Returns:

GstFlowReturn

setAverageBitrate

public void setAverageBitrate(int bitrate)

Optionally sets the average bitrate detected in media (if non-zero), e.g. based on metadata, as it will be posted to the application.

By default, announced average bitrate is estimated. The average bitrate is used to estimate the total duration of the stream and to estimate a seek position, if there's no index and the format is syncable (see gst_base_parse_set_syncable()).

Parameters:

bitrate - average bitrate in bits/second

setDuration

public void setDuration(Format fmt,
 long duration,
 int interval)

Sets the duration of the currently playing media. Subclass can use this when it is able to determine duration and/or notices a change in the media duration. Alternatively, if interval is non-zero (default), then stream duration is determined based on estimated bitrate, and updated every interval frames.

Parameters:

fmt - GstFormat.

duration - duration value.

interval - how often to update the duration estimate based on bitrate, or 0.

setFrameRate

public void setFrameRate(int fpsNum,
 int fpsDen,
 int leadIn,
 int leadOut)

If frames per second is configured, parser can take care of buffer duration and timestamping. When performing segment clipping, or seeking to a specific location, a corresponding decoder might need an initial leadIn and a following leadOut number of frames to ensure the desired segment is entirely filled upon decoding.

Parameters:

fpsNum - frames per second (numerator).

fpsDen - frames per second (denominator).

leadIn - frames needed before a segment for subsequent decode

leadOut - frames needed after a segment

setHasTimingInfo

public void setHasTimingInfo(boolean hasTiming)

Set if frames carry timing information which the subclass can (generally) parse and provide. In particular, intrinsic (rather than estimated) time can be obtained following a seek.

Parameters:

hasTiming - whether frames carry timing information

setInferTs

public void setInferTs(boolean inferTs)

By default, the base class might try to infer PTS from DTS and vice versa. While this is generally correct for audio data, it may not be otherwise. Sub-classes implementing such formats should disable timestamp inferring.

Parameters:

inferTs - true if parser should infer DTS/PTS from each other

setLatency

public void setLatency(ClockTime minLatency,
 ClockTime maxLatency)

Sets the minimum and maximum (which may likely be equal) latency introduced by the parsing process. If there is such a latency, which depends on the particular parsing of the format, it typically corresponds to 1 frame duration.

If the provided values changed from previously provided ones, this will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.

Parameters:

minLatency - minimum parse latency

maxLatency - maximum parse latency

setMinFrameSize

public void setMinFrameSize(int minSize)

Subclass can use this function to tell the base class that it needs to be given buffers of at least minSize bytes.

Parameters:

minSize - Minimum size in bytes of the data that this base class should give to subclass.

setPassthrough

public void setPassthrough(boolean passthrough)

Set if the nature of the format or configuration does not allow (much) parsing, and the parser should operate in passthrough mode (which only applies when operating in push mode). That is, incoming buffers are pushed through unmodified, i.e. no GstBaseParseClass::handle_frame will be invoked, but GstBaseParseClass::pre_push_frame will still be invoked, so subclass can perform as much or as little is appropriate for passthrough semantics in GstBaseParseClass::pre_push_frame.

Parameters:

passthrough - true if parser should run in passthrough mode

setPtsInterpolation

public void setPtsInterpolation(boolean ptsInterpolate)

By default, the base class will guess PTS timestamps using a simple interpolation (previous timestamp + duration), which is incorrect for data streams with reordering, where PTS can go backward. Sub-classes implementing such formats should disable PTS interpolation.

Parameters:

ptsInterpolate - true if parser should interpolate PTS timestamps

setSyncable

public void setSyncable(boolean syncable)

Set if frame starts can be identified. This is set by default and determines whether seeking based on bitrate averages is possible for a format/stream.

Parameters:

syncable - set if frame starts can be identified

setTsAtOffset

public void setTsAtOffset(long offset)

This function should only be called from a handleFrame implementation.

GstBaseParse creates initial timestamps for frames by using the last timestamp seen in the stream before the frame starts. In certain cases, the correct timestamps will occur in the stream after the start of the frame, but before the start of the actual picture data. This function can be used to set the timestamps based on the offset into the frame data that the picture starts.

Parameters:

offset - offset into current buffer

convert

protected boolean convert(Format srcFormat,
 long srcValue,
 Format destFormat,
 MemorySegment destValue)

Optional. Convert between formats.

detect

protected FlowReturn detect(Buffer buffer)

Optional. Called until it doesn't return GST_FLOW_OK anymore for the first buffers. Can be used by the subclass to detect the stream format.

getSinkCaps

protected Caps getSinkCaps(Caps filter)

Optional. Allows the subclass to do its own sink get caps if needed.

handleFrame

protected FlowReturn handleFrame(BaseParseFrame frame,
 Out<Integer> skipsize)

Parses the input data into valid frames as defined by subclass which should be passed to gst_base_parse_finish_frame(). The frame's input buffer is guaranteed writable, whereas the input frame ownership is held by caller (so subclass should make a copy if it needs to hang on). Input buffer (data) is provided by baseclass with as much metadata set as possible by baseclass according to upstream information and/or subclass settings, though subclass may still set buffer timestamp and duration if desired.

prePushFrame

protected FlowReturn prePushFrame(BaseParseFrame frame)

Optional. Called just prior to pushing a frame (after any pending events have been sent) to give subclass a chance to perform additional actions at this time (e.g. tag sending) or to decide whether this buffer should be dropped or not (e.g. custom segment clipping).

setSinkCaps

protected boolean setSinkCaps(Caps caps)

Optional. Allows the subclass to be notified of the actual caps set.

sinkEvent

protected boolean sinkEvent(Event event)

Optional. Event handler on the sink pad. This function should chain up to the parent implementation to let the default handler run.

sinkQuery

protected boolean sinkQuery(Query query)

Optional. Query handler on the sink pad. This function should chain up to the parent implementation to let the default handler run (Since: 1.2)

srcEvent

protected boolean srcEvent(Event event)

Optional. Event handler on the source pad. Should chain up to the parent to let the default handler run.

srcQuery

protected boolean srcQuery(Query query)

Optional. Query handler on the source pad. Should chain up to the parent to let the default handler run (Since: 1.2)

start

protected boolean start()

Optional. Called when the element starts processing. Allows opening external resources.

stop

protected boolean stop()

Optional. Called when the element stops processing. Allows closing external resources.

builder

public static BaseParse.Builder<? extends BaseParse.Builder> builder()

A BaseParse.Builder object constructs a BaseParse with the specified properties. Use the various set...() methods to set properties, and finish construction with BaseParse.Builder.build().