Package org.freedesktop.gstreamer.audio

Class AudioDecoder

java.lang.Object

io.github.jwharm.javagi.base.ProxyInstance

org.gnome.gobject.TypeInstance

org.gnome.gobject.GObject

org.gnome.gobject.InitiallyUnowned

org.freedesktop.gstreamer.gst.GstObject

org.freedesktop.gstreamer.gst.Element

org.freedesktop.gstreamer.audio.AudioDecoder

All Implemented Interfaces:

Proxy

Direct Known Subclasses:

AudioDecoder.AudioDecoderImpl

@Generated("io.github.jwharm.JavaGI")
public abstract class AudioDecoder
extends Element

This base class is for audio decoders turning encoded data into raw audio samples.

GstAudioDecoder and subclass should cooperate as follows.

Configuration

• Initially, GstAudioDecoder calls start when the decoder element is activated, which allows subclass to perform any global setup. Base class (context) parameters can already be set according to subclass capabilities (or possibly upon receive more information in subsequent setFormat).
• GstAudioDecoder calls setFormat to inform subclass of the format of input audio data that it is about to receive. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.
• GstAudioDecoder calls stop at end of all processing.

As of configuration stage, and throughout processing, GstAudioDecoder provides various (context) parameters, e.g. describing the format of output audio data (valid when output caps have been set) or current parsing state. Conversely, subclass can and should configure context to inform base class of its expectation w.r.t. buffer handling.

Data processing

• Base class gathers input data, and optionally allows subclass to parse this into subsequently manageable (as defined by subclass) chunks. Such chunks are subsequently referred to as 'frames', though they may or may not correspond to 1 (or more) audio format frame.
• Input frame is provided to subclass' handleFrame.
• If codec processing results in decoded data, subclass should call gstAudioDecoderFinishFrame to have decoded data pushed downstream.
• Just prior to actually pushing a buffer downstream, it is passed to prePush. Subclass should either use this callback to arrange for additional downstream pushing or otherwise ensure such custom pushing occurs after at least a method call has finished since setting src pad caps.
• During the parsing process GstAudioDecoderClass will handle both srcpad and sinkpad events. Sink events will be passed to subclass if event callback has been provided.

Shutdown phase

• GstAudioDecoder class calls stop to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' setFormat function, though it might be delayed until calling gstAudioDecoderFinishFrame.

In summary, above process should have subclass concentrating on codec data processing while leaving other matters to base class, such as most notably timestamp handling. While it may exert more control in this area (see e.g. prePush), it is very much not recommended.

In particular, base class will try to arrange for perfect output timestamps as much as possible while tracking upstream timestamps. To this end, if deviation between the next ideal expected perfect timestamp and upstream exceeds GstAudioDecoder:tolerance, then resync to upstream occurs (which would happen always if the tolerance mechanism is disabled).

In non-live pipelines, baseclass can also (configurably) arrange for output buffer aggregation which may help to redue large(r) numbers of small(er) buffers being pushed and processed downstream. Note that this feature is only available if the buffer layout is interleaved. For planar buffers, the decoder implementation is fully responsible for the output buffer size.

On the other hand, it should be noted that baseclass only provides limited seeking support (upon explicit subclass request), as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates.

Things that subclass need to take care of:

• Provide pad templates
• Set source pad caps when appropriate
• Set user-configurable properties to sane defaults for format and implementing codec at hand, and convey some subclass capabilities and expectations in context.

• Accept data in handleFrame and provide encoded results to gstAudioDecoderFinishFrame. If it is prepared to perform PLC, it should also accept NULL data in handleFrame and provide for data for indicated duration.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AudioDecoder.AudioDecoderClass	Subclasses can override any of the available virtual methods or not, as needed.
static class	AudioDecoder.AudioDecoderImpl	The AudioDecoderImpl type represents a native instance of the abstract AudioDecoder class.
static class	AudioDecoder.Builder<B extends AudioDecoder.Builder<B>>	Inner class implementing a builder pattern to construct a GObject with properties.

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.Element

Element.ElementClass, Element.ElementImpl, Element.NoMorePadsCallback, Element.PadAddedCallback, Element.PadRemovedCallback

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.GstObject

GstObject.DeepNotifyCallback, GstObject.ObjectClass, GstObject.ObjectImpl

Nested classes/interfaces inherited from class org.gnome.gobject.InitiallyUnowned

InitiallyUnowned.InitiallyUnownedClass

Nested classes/interfaces inherited from class org.gnome.gobject.GObject

GObject.NotifyCallback

Constructor Summary

Constructors

Constructor	Description
AudioDecoder(MemorySegment address)	Create a AudioDecoder proxy instance for the provided memory address.

Method Summary

Modifier and Type	Method	Description
Buffer	allocateOutputBuffer(long size)	Helper function that allocates a buffer to hold an audio frame for this AudioDecoder's current output format.
protected AudioDecoder	asParent()	Returns this instance as if it were its parent type.
static AudioDecoder.Builder<? extends AudioDecoder.Builder>	builder()	A AudioDecoder.Builder object constructs a AudioDecoder with the specified properties.
protected boolean	close()	Optional.
protected boolean	decideAllocation(Query query)	Optional.
FlowReturn	finishFrame(@Nullable Buffer buf, int frames)	Collects decoded data and pushes it downstream.
FlowReturn	finishSubframe(@Nullable Buffer buf)	Collects decoded data and pushes it downstream.
protected void	flush(boolean hard)	Optional.
void	getAllocator(@Nullable Out<Allocator> allocator, @Nullable AllocationParams params)	Lets GstAudioDecoder sub-classes to know the memory allocator used by the base class and its params.
AudioInfo	getAudioInfo()
protected Caps	getcaps(Caps filter)	Optional.
int	getDelay()
boolean	getDrainable()	Queries decoder drain handling.
int	getEstimateRate()
void	getLatency(@Nullable ClockTime min, @Nullable ClockTime max)	Sets the variables pointed to by min and max to the currently configured latency.
int	getMaxErrors()
static MemoryLayout	getMemoryLayout()	The memory layout of the native struct.
ClockTime	getMinLatency()	Queries decoder's latency aggregation.
boolean	getNeedsFormat()	Queries decoder required format handling.
void	getParseState(@Nullable Out<Boolean> sync, @Nullable Out<Boolean> eos)	Return current parsing (sync and eos) state.
boolean	getPlc()	Queries decoder packet loss concealment handling.
int	getPlcAware()
ClockTime	getTolerance()	Queries current audio jitter tolerance threshold.
static Type	getType()	Get the GType of the AudioDecoder class
protected FlowReturn	handleFrame(Buffer buffer)	Provides input data (or NULL to clear any remaining data) to subclass.
void	mergeTags(@Nullable TagList tags, TagMergeMode mode)	Sets the audio decoder tags and how they should be merged with any upstream stream tags.
boolean	negotiate()	Negotiate with downstream elements to currently configured GstAudioInfo.
protected boolean	open()	Optional.
protected FlowReturn	parse(Adapter adapter, Out<Integer> offset, Out<Integer> length)
protected FlowReturn	prePush(Buffer[] buffer)	Optional.
protected boolean	proposeAllocation(Query query)	Optional.
Caps	proxyGetcaps(@Nullable Caps caps, @Nullable Caps filter)	Returns caps that express caps (or sink template caps if caps == NULL) restricted to rate/channels/...
void	setAllocationCaps(@Nullable Caps allocationCaps)	Sets a caps in allocation query which are different from the set pad's caps.
void	setDrainable(boolean enabled)	Configures decoder drain handling.
void	setEstimateRate(boolean enabled)	Allows baseclass to perform byte to time estimated conversion.
protected boolean	setFormat(Caps caps)	Notifies subclass of incoming data format (caps).
void	setLatency(ClockTime min, ClockTime max)	Sets decoder latency.
void	setMaxErrors(int num)	Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error.
void	setMinLatency(ClockTime num)	Sets decoder minimum aggregation latency.
void	setNeedsFormat(boolean enabled)	Configures decoder format needs.
boolean	setOutputCaps(Caps caps)	Configure output caps on the srcpad of this AudioDecoder.
boolean	setOutputFormat(AudioInfo info)	Configure output info on the srcpad of this AudioDecoder.
void	setPlc(boolean enabled)	Enable or disable decoder packet loss concealment, provided subclass and codec are capable and allow handling plc.
void	setPlcAware(boolean plc)	Indicates whether or not subclass handles packet loss concealment (plc).
void	setTolerance(ClockTime tolerance)	Configures decoder audio jitter tolerance threshold.
void	setUseDefaultPadAcceptcaps(boolean use)	Lets GstAudioDecoder sub-classes decide if they want the sink pad to use the default pad query handler to reply to accept-caps queries.
protected boolean	sinkEvent(Event event)	Optional.
protected boolean	sinkQuery(Query query)	Optional.
protected boolean	srcEvent(Event event)	Optional.
protected boolean	srcQuery(Query query)	Optional.
protected boolean	start()	Optional.
protected boolean	stop()	Optional.
protected boolean	transformMeta(Buffer outbuf, Meta meta, Buffer inbuf)	Optional.

Methods inherited from class org.freedesktop.gstreamer.gst.Element

abortState, addPad, addPropertyDeepNotifyWatch, addPropertyNotifyWatch, callAsync, changeState, continueState, createAllPads, decorateStreamId, decorateStreamIdPrintf, emitNoMorePads, emitPadAdded, emitPadRemoved, foreachPad, foreachSinkPad, foreachSrcPad, getBaseTime, getBus, getClock, getCompatiblePad, getCompatiblePadTemplate, getContext, getContexts, getContextUnlocked, getCurrentClockTime, getCurrentRunningTime, getFactory, getMetadata, getPadTemplate, getPadTemplateList, getRequestPad, getStartTime, getState, getStaticPad, isLockedState, iteratePads, iterateSinkPads, iterateSrcPads, link, linkFiltered, linkMany, linkPads, linkPadsFiltered, linkPadsFull, linkPadsFull, lostState, makeFromUri, messageFull, messageFull, messageFullWithDetails, messageFullWithDetails, noMorePads, onNoMorePads, onPadAdded, onPadRemoved, padAdded, padRemoved, postMessage, provideClock, query, queryConvert, queryDuration, queryPosition, register, releasePad, releaseRequestPad, removePad, removePropertyNotifyWatch, requestPad, requestPadSimple, seek, seek, seekSimple, seekSimple, sendEvent, setBaseTime, setBus, setClock, setContext, setLockedState, setStartTime, setState, stateChanged, stateChangeReturnGetName, stateGetName, syncStateWithParent, typeSetSkipDocumentation, unlink, unlinkMany, unlinkPads

Methods inherited from class org.freedesktop.gstreamer.gst.GstObject

addControlBinding, checkUniqueness, deepNotify, defaultDeepNotify, defaultError, emitDeepNotify, getControlBinding, getControlRate, getGValueArray, getName, getParent, getPathString, getValue, getValueArray, hasActiveControlBindings, hasAncestor, hasAsAncestor, hasAsParent, onDeepNotify, ref, refSink, removeControlBinding, replace, setControlBindingDisabled, setControlBindingsDisabled, setControlRate, setName, setParent, suggestNextSync, syncValues, unparent, unref

Methods inherited from class org.gnome.gobject.GObject

addToggleRef, addWeakPointer, bindProperty, bindProperty, bindProperty, bindPropertyFull, bindPropertyFull, bindPropertyWithClosures, bindPropertyWithClosures, compatControl, connect, connect, connect, constructed, disconnect, dispatchPropertiesChanged, dispose, dupData, dupQdata, emit, emitNotify, finalize_, forceFloating, freezeNotify, get, getData, getProperty, getProperty, getProperty, getQdata, getv, interfaceFindProperty, interfaceInstallProperty, interfaceListProperties, isFloating, newInstance, newInstance, newInstance, newInstance, newv, notify_, notify_, notifyByPspec, onNotify, refSink, removeToggleRef, removeWeakPointer, replaceData, replaceQdata, runDispose, set, setData, setDataFull, setProperty, setProperty, setProperty, setQdata, setQdataFull, setv, stealData, stealQdata, takeRef, thawNotify, watchClosure, weakRef, weakUnref, withProperties

Methods inherited from class org.gnome.gobject.TypeInstance

callParent, callParent, getPrivate, readGClass, writeGClass

Methods inherited from class io.github.jwharm.javagi.base.ProxyInstance

equals, handle, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Details

AudioDecoder

public AudioDecoder(MemorySegment address)

Create a AudioDecoder proxy instance for the provided memory address.

Parameters:

address - the memory address of the native object

Method Details

getType

public static Type getType()

Get the GType of the AudioDecoder class

Returns:

the GType

getMemoryLayout

public static MemoryLayout getMemoryLayout()

The memory layout of the native struct.

Returns:

the memory layout

asParent

protected AudioDecoder asParent()

Returns this instance as if it were its parent type. This is mostly synonymous to the Java super keyword, but will set the native typeclass function pointers to the parent type. When overriding a native virtual method in Java, "chaining up" with super.methodName() doesn't work, because it invokes the overridden function pointer again. To chain up, call asParent().methodName(). This will call the native function pointer of this virtual method in the typeclass of the parent type.

Overrides:

asParent in class Element

allocateOutputBuffer

public Buffer allocateOutputBuffer(long size)

Helper function that allocates a buffer to hold an audio frame for this AudioDecoder's current output format.

Parameters:

size - size of the buffer

Returns:

allocated buffer

finishFrame

public FlowReturn finishFrame(@Nullable
 @Nullable Buffer buf,
 int frames)

Collects decoded data and pushes it downstream.

buf may be NULL in which case the indicated number of frames are discarded and considered to have produced no output (e.g. lead-in or setup frames). Otherwise, source pad caps must be set when it is called with valid data in buf.

Note that a frame received in GstAudioDecoderClass.handle_frame() may be invalidated by a call to this function.

Parameters:

buf - decoded data

frames - number of decoded frames represented by decoded data

Returns:

a GstFlowReturn that should be escalated to caller (of caller)

finishSubframe

public FlowReturn finishSubframe(@Nullable
 @Nullable Buffer buf)

Collects decoded data and pushes it downstream. This function may be called multiple times for a given input frame.

buf may be NULL in which case it is assumed that the current input frame is finished. This is equivalent to calling gst_audio_decoder_finish_subframe() with a NULL buffer and frames=1 after having pushed out all decoded audio subframes using this function.

When called with valid data in buf the source pad caps must have been set already.

Note that a frame received in GstAudioDecoderClass.handle_frame() may be invalidated by a call to this function.

Parameters:

buf - decoded data

Returns:

a GstFlowReturn that should be escalated to caller (of caller)

getAllocator

public void getAllocator(@Nullable
 @Nullable Out<Allocator> allocator,
 @Nullable
 @Nullable AllocationParams params)

Lets GstAudioDecoder sub-classes to know the memory allocator used by the base class and its params.

Unref the allocator after use it.

Parameters:

allocator - the GstAllocator used

params - the GstAllocationParams of allocator

getAudioInfo

public AudioInfo getAudioInfo()

getDelay

public int getDelay()

getDrainable

public boolean getDrainable()

Queries decoder drain handling.

Returns:

TRUE if drainable handling is enabled.

MT safe.

getEstimateRate

public int getEstimateRate()

getLatency

public void getLatency(@Nullable
 @Nullable ClockTime min,
 @Nullable
 @Nullable ClockTime max)

Sets the variables pointed to by min and max to the currently configured latency.

Parameters:

min - a pointer to storage to hold minimum latency

max - a pointer to storage to hold maximum latency

getMaxErrors

public int getMaxErrors()

getMinLatency

public ClockTime getMinLatency()

Queries decoder's latency aggregation.

Returns:

aggregation latency.

MT safe.

getNeedsFormat

public boolean getNeedsFormat()

Queries decoder required format handling.

Returns:

TRUE if required format handling is enabled.

MT safe.

getParseState

public void getParseState(@Nullable
 @Nullable Out<Boolean> sync,
 @Nullable
 @Nullable Out<Boolean> eos)

Return current parsing (sync and eos) state.

Parameters:

sync - a pointer to a variable to hold the current sync state

eos - a pointer to a variable to hold the current eos state

getPlc

public boolean getPlc()

Queries decoder packet loss concealment handling.

Returns:

TRUE if packet loss concealment is enabled.

MT safe.

getPlcAware

public int getPlcAware()

getTolerance

public ClockTime getTolerance()

Queries current audio jitter tolerance threshold.

Returns:

decoder audio jitter tolerance threshold.

MT safe.

mergeTags

public void mergeTags(@Nullable
 @Nullable TagList tags,
 TagMergeMode mode)

Sets the audio decoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_audio_decoder_merge_tags().

Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.

Parameters:

tags - a GstTagList to merge, or NULL

mode - the GstTagMergeMode to use, usually GST_TAG_MERGE_REPLACE

negotiate

public boolean negotiate()

Negotiate with downstream elements to currently configured GstAudioInfo. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.

Returns:

true if the negotiation succeeded, else false.

proxyGetcaps

public Caps proxyGetcaps(@Nullable
 @Nullable Caps caps,
 @Nullable
 @Nullable Caps filter)

Returns caps that express caps (or sink template caps if caps == NULL) restricted to rate/channels/... combinations supported by downstream elements.

Parameters:

caps - initial caps

filter - filter caps

Returns:

a GstCaps owned by caller

setAllocationCaps

public void setAllocationCaps(@Nullable
 @Nullable Caps allocationCaps)

Sets a caps in allocation query which are different from the set pad's caps. Use this function before calling gst_audio_decoder_negotiate(). Setting to null the allocation query will use the caps from the pad.

Parameters:

allocationCaps - a GstCaps or null

setDrainable

public void setDrainable(boolean enabled)

Configures decoder drain handling. If drainable, subclass might be handed a NULL buffer to have it return any leftover decoded data. Otherwise, it is not considered so capable and will only ever be passed real data.

MT safe.

Parameters:

enabled - new state

setEstimateRate

public void setEstimateRate(boolean enabled)

Allows baseclass to perform byte to time estimated conversion.

Parameters:

enabled - whether to enable byte to time conversion

setLatency

public void setLatency(ClockTime min,
 ClockTime max)

Sets decoder latency. If the provided values changed from previously provided ones, this will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.

Parameters:

min - minimum latency

max - maximum latency

setMaxErrors

public void setMaxErrors(int num)

Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error. You can set -1 for never returning fatal errors. Default is set to GST_AUDIO_DECODER_MAX_ERRORS.

Parameters:

num - max tolerated errors

setMinLatency

public void setMinLatency(ClockTime num)

Sets decoder minimum aggregation latency.

MT safe.

Parameters:

num - new minimum latency

setNeedsFormat

public void setNeedsFormat(boolean enabled)

Configures decoder format needs. If enabled, subclass needs to be negotiated with format caps before it can process any data. It will then never be handed any data before it has been configured. Otherwise, it might be handed data without having been configured and is then expected being able to do so either by default or based on the input data.

MT safe.

Parameters:

enabled - new state

setOutputCaps

public boolean setOutputCaps(Caps caps)

Configure output caps on the srcpad of this AudioDecoder. Similar to gst_audio_decoder_set_output_format(), but allows subclasses to specify output caps that can't be expressed via GstAudioInfo e.g. caps that have caps features.

Parameters:

caps - (fixed) GstCaps

Returns:

true on success.

setOutputFormat

public boolean setOutputFormat(AudioInfo info)

Configure output info on the srcpad of this AudioDecoder.

Parameters:

info - GstAudioInfo

Returns:

true on success.

setPlc

public void setPlc(boolean enabled)

Enable or disable decoder packet loss concealment, provided subclass and codec are capable and allow handling plc.

MT safe.

Parameters:

enabled - new state

setPlcAware

public void setPlcAware(boolean plc)

Indicates whether or not subclass handles packet loss concealment (plc).

Parameters:

plc - new plc state

setTolerance

public void setTolerance(ClockTime tolerance)

Configures decoder audio jitter tolerance threshold.

MT safe.

Parameters:

tolerance - new tolerance

setUseDefaultPadAcceptcaps

public void setUseDefaultPadAcceptcaps(boolean use)

Lets GstAudioDecoder sub-classes decide if they want the sink pad to use the default pad query handler to reply to accept-caps queries.

By setting this to true it is possible to further customize the default handler with GST_PAD_SET_ACCEPT_INTERSECT and GST_PAD_SET_ACCEPT_TEMPLATE

Parameters:

use - if the default pad accept-caps query handling should be used

close

protected boolean close()

Optional. Called when the element changes to GST_STATE_NULL. Allows closing external resources.

decideAllocation

protected boolean decideAllocation(Query query)

Optional. Setup the allocation parameters for allocating output buffers. The passed in query contains the result of the downstream allocation query. Subclasses should chain up to the parent implementation to invoke the default handler.

flush

protected void flush(boolean hard)

Optional. Instructs subclass to clear any codec caches and discard any pending samples and not yet returned decoded data. hard indicates whether a FLUSH is being processed, or otherwise a DISCONT (or conceptually similar).

getcaps

protected Caps getcaps(Caps filter)

Optional. Allows for a custom sink getcaps implementation. If not implemented, default returns gst_audio_decoder_proxy_getcaps applied to sink template caps.

handleFrame

protected FlowReturn handleFrame(Buffer buffer)

Provides input data (or NULL to clear any remaining data) to subclass. Input data ref management is performed by base class, subclass should not care or intervene, and input data is only valid until next call to base class, most notably a call to gst_audio_decoder_finish_frame().

open

protected boolean open()

Optional. Called when the element changes to GST_STATE_READY. Allows opening external resources.

parse

protected FlowReturn parse(Adapter adapter,
 Out<Integer> offset,
 Out<Integer> length)

prePush

protected FlowReturn prePush(Buffer[] buffer)

Optional. Called just prior to pushing (encoded data) buffer downstream. Subclass has full discretionary access to buffer, and a not OK flow return will abort downstream pushing.

proposeAllocation

protected boolean proposeAllocation(Query query)

Optional. Propose buffer allocation parameters for upstream elements. Subclasses should chain up to the parent implementation to invoke the default handler.

setFormat

protected boolean setFormat(Caps caps)

Notifies subclass of incoming data format (caps).

sinkEvent

protected boolean sinkEvent(Event event)

Optional. Event handler on the sink pad. Subclasses should chain up to the parent implementation to invoke the default handler.

sinkQuery

protected boolean sinkQuery(Query query)

Optional. Query handler on the sink pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.6

srcEvent

protected boolean srcEvent(Event event)

Optional. Event handler on the src pad. Subclasses should chain up to the parent implementation to invoke the default handler.

srcQuery

protected boolean srcQuery(Query query)

Optional. Query handler on the source pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.6

start

protected boolean start()

Optional. Called when the element starts processing. Allows opening external resources.

stop

protected boolean stop()

Optional. Called when the element stops processing. Allows closing external resources.

transformMeta

protected boolean transformMeta(Buffer outbuf,
 Meta meta,
 Buffer inbuf)

Optional. Transform the metadata on the input buffer to the output buffer. By default this method copies all meta without tags and meta with only the "audio" tag. subclasses can implement this method and return true if the metadata is to be copied. Since: 1.6

builder

public static AudioDecoder.Builder<? extends AudioDecoder.Builder> builder()

A AudioDecoder.Builder object constructs a AudioDecoder with the specified properties. Use the various set...() methods to set properties, and finish construction with AudioDecoder.Builder.build().