Package org.freedesktop.gstreamer.audio

Class AudioEncoder

java.lang.Object

io.github.jwharm.javagi.base.ProxyInstance

org.gnome.gobject.TypeInstance

org.gnome.gobject.GObject

org.gnome.gobject.InitiallyUnowned

org.freedesktop.gstreamer.gst.GstObject

org.freedesktop.gstreamer.gst.Element

org.freedesktop.gstreamer.audio.AudioEncoder

All Implemented Interfaces:

Proxy, Preset

Direct Known Subclasses:

AudioEncoder.AudioEncoderImpl

@Generated("io.github.jwharm.JavaGI")
public abstract class AudioEncoder
extends Element
implements Preset

This base class is for audio encoders turning raw audio samples into encoded audio data.

GstAudioEncoder and subclass should cooperate as follows.

Configuration

• Initially, GstAudioEncoder calls start when the encoder element is activated, which allows subclass to perform any global setup.

• GstAudioEncoder calls setFormat to inform subclass of the format of input audio data that it is about to receive. Subclass should setup for encoding and configure various base class parameters appropriately, notably those directing desired input data handling. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.

• GstAudioEncoder calls stop at end of all processing.

As of configuration stage, and throughout processing, GstAudioEncoder maintains various parameters that provide required context, e.g. describing the format of input audio data. Conversely, subclass can and should configure these context parameters to inform base class of its expectation w.r.t. buffer handling.

Data processing

• Base class gathers input sample data (as directed by the context's frame_samples and frame_max) and provides this to subclass' handleFrame.
• If codec processing results in encoded data, subclass should call gst_audio_encoder_finish_frame() to have encoded data pushed downstream. Alternatively, it might also call gst_audio_encoder_finish_frame() (with a NULL buffer and some number of dropped samples) to indicate dropped (non-encoded) samples.
• Just prior to actually pushing a buffer downstream, it is passed to prePush.
• During the parsing process GstAudioEncoderClass will handle both srcpad and sinkpad events. Sink events will be passed to subclass if event callback has been provided.

Shutdown phase

• GstAudioEncoder class calls stop to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' setFormat function, though it might be delayed until calling gstAudioEncoderFinishFrame.

In summary, above process should have subclass concentrating on codec data processing while leaving other matters to base class, such as most notably timestamp handling. While it may exert more control in this area (see e.g. prePush), it is very much not recommended.

In particular, base class will either favor tracking upstream timestamps (at the possible expense of jitter) or aim to arrange for a perfect stream of output timestamps, depending on GstAudioEncoder:perfect-timestamp. However, in the latter case, the input may not be so perfect or ideal, which is handled as follows. An input timestamp is compared with the expected timestamp as dictated by input sample stream and if the deviation is less than GstAudioEncoder:tolerance, the deviation is discarded. Otherwise, it is considered a discontuinity and subsequent output timestamp is resynced to the new position after performing configured discontinuity processing. In the non-perfect-timestamp case, an upstream variation exceeding tolerance only leads to marking DISCONT on subsequent outgoing (while timestamps are adjusted to upstream regardless of variation). While DISCONT is also marked in the perfect-timestamp case, this one optionally (see GstAudioEncoder:hard-resync) performs some additional steps, such as clipping of (early) input samples or draining all currently remaining input data, depending on the direction of the discontuinity.

If perfect timestamps are arranged, it is also possible to request baseclass (usually set by subclass) to provide additional buffer metadata (in OFFSET and OFFSET_END) fields according to granule defined semantics currently needed by oggmux. Specifically, OFFSET is set to granulepos (= sample count including buffer) and OFFSET_END to corresponding timestamp (as determined by same sample count and sample rate).

Things that subclass need to take care of:

• Provide pad templates
• Set source pad caps when appropriate
• Inform base class of buffer processing needs using context's frame_samples and frame_bytes.
• Set user-configurable properties to sane defaults for format and implementing codec at hand, e.g. those controlling timestamp behaviour and discontinuity processing.
• Accept data in handleFrame and provide encoded results to gst_audio_encoder_finish_frame().

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AudioEncoder.AudioEncoderClass	Subclasses can override any of the available virtual methods or not, as needed.
static class	AudioEncoder.AudioEncoderImpl	The AudioEncoderImpl type represents a native instance of the abstract AudioEncoder class.
static class	AudioEncoder.Builder<B extends AudioEncoder.Builder<B>>	Inner class implementing a builder pattern to construct a GObject with properties.

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.Element

Element.ElementClass, Element.ElementImpl, Element.NoMorePadsCallback, Element.PadAddedCallback, Element.PadRemovedCallback

Nested classes/interfaces inherited from class org.freedesktop.gstreamer.gst.GstObject

GstObject.DeepNotifyCallback, GstObject.ObjectClass, GstObject.ObjectImpl

Nested classes/interfaces inherited from class org.gnome.gobject.InitiallyUnowned

InitiallyUnowned.InitiallyUnownedClass

Nested classes/interfaces inherited from class org.gnome.gobject.GObject

GObject.NotifyCallback

Nested classes/interfaces inherited from interface org.freedesktop.gstreamer.gst.Preset

Preset.PresetImpl, Preset.PresetInterface

Constructor Summary

Constructors

Constructor	Description
AudioEncoder(MemorySegment address)	Create a AudioEncoder proxy instance for the provided memory address.

Method Summary

Modifier and Type	Method	Description
Buffer	allocateOutputBuffer(long size)	Helper function that allocates a buffer to hold an encoded audio frame for this AudioEncoder's current output format.
protected AudioEncoder	asParent()	Returns this instance as if it were its parent type.
static AudioEncoder.Builder<? extends AudioEncoder.Builder>	builder()	A AudioEncoder.Builder object constructs a AudioEncoder with the specified properties.
protected boolean	close()	Optional.
protected boolean	decideAllocation(Query query)	Optional.
FlowReturn	finishFrame(@Nullable Buffer buffer, int samples)	Collects encoded data and pushes encoded data downstream.
protected void	flush()	Optional.
void	getAllocator(@Nullable Out<Allocator> allocator, @Nullable AllocationParams params)	Lets GstAudioEncoder sub-classes to know the memory allocator used by the base class and its params.
AudioInfo	getAudioInfo()
protected Caps	getcaps(Caps filter)	Optional.
boolean	getDrainable()	Queries encoder drain handling.
int	getFrameMax()
int	getFrameSamplesMax()
int	getFrameSamplesMin()
boolean	getHardMin()	Queries encoder hard minimum handling.
boolean	getHardResync()
void	getLatency(@Nullable ClockTime min, @Nullable ClockTime max)	Sets the variables pointed to by min and max to the currently configured latency.
int	getLookahead()
boolean	getMarkGranule()	Queries if the encoder will handle granule marking.
static MemoryLayout	getMemoryLayout()	The memory layout of the native struct.
boolean	getPerfectTimestamp()	Queries encoder perfect timestamp behaviour.
ClockTime	getTolerance()	Queries current audio jitter tolerance threshold.
static Type	getType()	Get the GType of the AudioEncoder class
protected FlowReturn	handleFrame(Buffer buffer)	Provides input samples (or NULL to clear any remaining data) according to directions as configured by the subclass using the API.
void	mergeTags(@Nullable TagList tags, TagMergeMode mode)	Sets the audio encoder tags and how they should be merged with any upstream stream tags.
boolean	negotiate()	Negotiate with downstream elements to currently configured GstCaps.
protected boolean	open()	Optional.
protected FlowReturn	prePush(Buffer[] buffer)	Optional.
protected boolean	proposeAllocation(Query query)	Optional.
Caps	proxyGetcaps(@Nullable Caps caps, @Nullable Caps filter)	Returns caps that express caps (or sink template caps if caps == NULL) restricted to channel/rate combinations supported by downstream elements (e.g.
void	setAllocationCaps(@Nullable Caps allocationCaps)	Sets a caps in allocation query which are different from the set pad's caps.
void	setDrainable(boolean enabled)	Configures encoder drain handling.
protected boolean	setFormat(AudioInfo info)	Notifies subclass of incoming data format.
void	setFrameMax(int num)	Sets max number of frames accepted at once (assumed minimally 1).
void	setFrameSamplesMax(int num)	Sets number of samples (per channel) subclass needs to be handed, at most or will be handed all available if 0.
void	setFrameSamplesMin(int num)	Sets number of samples (per channel) subclass needs to be handed, at least or will be handed all available if 0.
void	setHardMin(boolean enabled)	Configures encoder hard minimum handling.
void	setHardResync(boolean enabled)
void	setHeaders(List<Buffer> headers)	Set the codec headers to be sent downstream whenever requested.
void	setLatency(ClockTime min, ClockTime max)	Sets encoder latency.
void	setLookahead(int num)	Sets encoder lookahead (in units of input rate samples)
void	setMarkGranule(boolean enabled)	Enable or disable encoder granule handling.
boolean	setOutputFormat(Caps caps)	Configure output caps on the srcpad of this AudioEncoder.
void	setPerfectTimestamp(boolean enabled)	Enable or disable encoder perfect output timestamp preference.
void	setTolerance(ClockTime tolerance)	Configures encoder audio jitter tolerance threshold.
protected boolean	sinkEvent(Event event)	Optional.
protected boolean	sinkQuery(Query query)	Optional.
protected boolean	srcEvent(Event event)	Optional.
protected boolean	srcQuery(Query query)	Optional.
protected boolean	start()	Optional.
protected boolean	stop()	Optional.
protected boolean	transformMeta(Buffer outbuf, Meta meta, Buffer inbuf)	Optional.

Methods inherited from class org.freedesktop.gstreamer.gst.Element

abortState, addPad, addPropertyDeepNotifyWatch, addPropertyNotifyWatch, callAsync, changeState, continueState, createAllPads, decorateStreamId, decorateStreamIdPrintf, emitNoMorePads, emitPadAdded, emitPadRemoved, foreachPad, foreachSinkPad, foreachSrcPad, getBaseTime, getBus, getClock, getCompatiblePad, getCompatiblePadTemplate, getContext, getContexts, getContextUnlocked, getCurrentClockTime, getCurrentRunningTime, getFactory, getMetadata, getPadTemplate, getPadTemplateList, getRequestPad, getStartTime, getState, getStaticPad, isLockedState, iteratePads, iterateSinkPads, iterateSrcPads, link, linkFiltered, linkMany, linkPads, linkPadsFiltered, linkPadsFull, linkPadsFull, lostState, makeFromUri, messageFull, messageFull, messageFullWithDetails, messageFullWithDetails, noMorePads, onNoMorePads, onPadAdded, onPadRemoved, padAdded, padRemoved, postMessage, provideClock, query, queryConvert, queryDuration, queryPosition, register, releasePad, releaseRequestPad, removePad, removePropertyNotifyWatch, requestPad, requestPadSimple, seek, seek, seekSimple, seekSimple, sendEvent, setBaseTime, setBus, setClock, setContext, setLockedState, setStartTime, setState, stateChanged, stateChangeReturnGetName, stateGetName, syncStateWithParent, typeSetSkipDocumentation, unlink, unlinkMany, unlinkPads

Methods inherited from class org.freedesktop.gstreamer.gst.GstObject

addControlBinding, checkUniqueness, deepNotify, defaultDeepNotify, defaultError, emitDeepNotify, getControlBinding, getControlRate, getGValueArray, getName, getParent, getPathString, getValue, getValueArray, hasActiveControlBindings, hasAncestor, hasAsAncestor, hasAsParent, onDeepNotify, ref, refSink, removeControlBinding, replace, setControlBindingDisabled, setControlBindingsDisabled, setControlRate, setName, setParent, suggestNextSync, syncValues, unparent, unref

Methods inherited from class org.gnome.gobject.GObject

addToggleRef, addWeakPointer, bindProperty, bindProperty, bindProperty, bindPropertyFull, bindPropertyFull, bindPropertyWithClosures, bindPropertyWithClosures, compatControl, connect, connect, connect, constructed, disconnect, dispatchPropertiesChanged, dispose, dupData, dupQdata, emit, emitNotify, finalize_, forceFloating, freezeNotify, get, getData, getProperty, getProperty, getProperty, getQdata, getv, interfaceFindProperty, interfaceInstallProperty, interfaceListProperties, isFloating, newInstance, newInstance, newInstance, newInstance, newv, notify_, notify_, notifyByPspec, onNotify, refSink, removeToggleRef, removeWeakPointer, replaceData, replaceQdata, runDispose, set, setData, setDataFull, setProperty, setProperty, setProperty, setQdata, setQdataFull, setv, stealData, stealQdata, takeRef, thawNotify, watchClosure, weakRef, weakUnref, withProperties

Methods inherited from class org.gnome.gobject.TypeInstance

callParent, callParent, getPrivate, readGClass, writeGClass

Methods inherited from class io.github.jwharm.javagi.base.ProxyInstance

equals, handle, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.freedesktop.gstreamer.gst.Preset

deletePreset, getMeta, getPresetNames, getPropertyNames, isEditable, loadPreset, renamePreset, savePreset, setMeta

Methods inherited from interface io.github.jwharm.javagi.base.Proxy

handle

Constructor Details

AudioEncoder

public AudioEncoder(MemorySegment address)

Create a AudioEncoder proxy instance for the provided memory address.

Parameters:

address - the memory address of the native object

Method Details

getType

public static Type getType()

Get the GType of the AudioEncoder class

Returns:

the GType

getMemoryLayout

public static MemoryLayout getMemoryLayout()

The memory layout of the native struct.

Returns:

the memory layout

asParent

protected AudioEncoder asParent()

Returns this instance as if it were its parent type. This is mostly synonymous to the Java super keyword, but will set the native typeclass function pointers to the parent type. When overriding a native virtual method in Java, "chaining up" with super.methodName() doesn't work, because it invokes the overridden function pointer again. To chain up, call asParent().methodName(). This will call the native function pointer of this virtual method in the typeclass of the parent type.

Overrides:

asParent in class Element

allocateOutputBuffer

public Buffer allocateOutputBuffer(long size)

Helper function that allocates a buffer to hold an encoded audio frame for this AudioEncoder's current output format.

Parameters:

size - size of the buffer

Returns:

allocated buffer

finishFrame

public FlowReturn finishFrame(@Nullable
 @Nullable Buffer buffer,
 int samples)

Collects encoded data and pushes encoded data downstream. Source pad caps must be set when this is called.

If samples < 0, then best estimate is all samples provided to encoder (subclass) so far. buf may be NULL, in which case next number of samples are considered discarded, e.g. as a result of discontinuous transmission, and a discontinuity is marked.

Note that samples received in GstAudioEncoderClass.handle_frame() may be invalidated by a call to this function.

Parameters:

buffer - encoded data

samples - number of samples (per channel) represented by encoded data

Returns:

a GstFlowReturn that should be escalated to caller (of caller)

getAllocator

public void getAllocator(@Nullable
 @Nullable Out<Allocator> allocator,
 @Nullable
 @Nullable AllocationParams params)

Lets GstAudioEncoder sub-classes to know the memory allocator used by the base class and its params.

Unref the allocator after use it.

Parameters:

allocator - the GstAllocator used

params - the GstAllocationParams of allocator

getAudioInfo

public AudioInfo getAudioInfo()

getDrainable

public boolean getDrainable()

Queries encoder drain handling.

Returns:

TRUE if drainable handling is enabled.

MT safe.

getFrameMax

public int getFrameMax()

getFrameSamplesMax

public int getFrameSamplesMax()

getFrameSamplesMin

public int getFrameSamplesMin()

getHardMin

public boolean getHardMin()

Queries encoder hard minimum handling.

Returns:

TRUE if hard minimum handling is enabled.

MT safe.

getHardResync

public boolean getHardResync()

getLatency

public void getLatency(@Nullable
 @Nullable ClockTime min,
 @Nullable
 @Nullable ClockTime max)

Sets the variables pointed to by min and max to the currently configured latency.

Parameters:

min - a pointer to storage to hold minimum latency

max - a pointer to storage to hold maximum latency

getLookahead

public int getLookahead()

getMarkGranule

public boolean getMarkGranule()

Queries if the encoder will handle granule marking.

Returns:

TRUE if granule marking is enabled.

MT safe.

getPerfectTimestamp

public boolean getPerfectTimestamp()

Queries encoder perfect timestamp behaviour.

Returns:

TRUE if perfect timestamp setting enabled.

MT safe.

getTolerance

public ClockTime getTolerance()

Queries current audio jitter tolerance threshold.

Returns:

encoder audio jitter tolerance threshold.

MT safe.

mergeTags

public void mergeTags(@Nullable
 @Nullable TagList tags,
 TagMergeMode mode)

Sets the audio encoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_audio_encoder_merge_tags().

Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.

MT safe.

Parameters:

tags - a GstTagList to merge, or NULL to unset previously-set tags

mode - the GstTagMergeMode to use, usually GST_TAG_MERGE_REPLACE

negotiate

public boolean negotiate()

Negotiate with downstream elements to currently configured GstCaps. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.

Returns:

true if the negotiation succeeded, else false.

proxyGetcaps

public Caps proxyGetcaps(@Nullable
 @Nullable Caps caps,
 @Nullable
 @Nullable Caps filter)

Returns caps that express caps (or sink template caps if caps == NULL) restricted to channel/rate combinations supported by downstream elements (e.g. muxers).

Parameters:

caps - initial caps

filter - filter caps

Returns:

a GstCaps owned by caller

setAllocationCaps

public void setAllocationCaps(@Nullable
 @Nullable Caps allocationCaps)

Sets a caps in allocation query which are different from the set pad's caps. Use this function before calling gst_audio_encoder_negotiate(). Setting to null the allocation query will use the caps from the pad.

Parameters:

allocationCaps - a GstCaps or null

setDrainable

public void setDrainable(boolean enabled)

Configures encoder drain handling. If drainable, subclass might be handed a NULL buffer to have it return any leftover encoded data. Otherwise, it is not considered so capable and will only ever be passed real data.

MT safe.

Parameters:

enabled - new state

setFrameMax

public void setFrameMax(int num)

Sets max number of frames accepted at once (assumed minimally 1). Requires frameSamplesMin and frameSamplesMax to be the equal.

Note: This value will be reset to 0 every time before GstAudioEncoderClass.set_format() is called.

Parameters:

num - number of frames

setFrameSamplesMax

public void setFrameSamplesMax(int num)

Sets number of samples (per channel) subclass needs to be handed, at most or will be handed all available if 0.

If an exact number of samples is required, gst_audio_encoder_set_frame_samples_min() must be called with the same number.

Note: This value will be reset to 0 every time before GstAudioEncoderClass.set_format() is called.

Parameters:

num - number of samples per frame

setFrameSamplesMin

public void setFrameSamplesMin(int num)

Sets number of samples (per channel) subclass needs to be handed, at least or will be handed all available if 0.

If an exact number of samples is required, gst_audio_encoder_set_frame_samples_max() must be called with the same number.

Note: This value will be reset to 0 every time before GstAudioEncoderClass.set_format() is called.

Parameters:

num - number of samples per frame

setHardMin

public void setHardMin(boolean enabled)

Configures encoder hard minimum handling. If enabled, subclass will never be handed less samples than it configured, which otherwise might occur near end-of-data handling. Instead, the leftover samples will simply be discarded.

MT safe.

Parameters:

enabled - new state

setHardResync

public void setHardResync(boolean enabled)

setHeaders

public void setHeaders(List<Buffer> headers)

Set the codec headers to be sent downstream whenever requested.

Parameters:

headers - a list of GstBuffer containing the codec header

setLatency

public void setLatency(ClockTime min,
 ClockTime max)

Sets encoder latency. If the provided values changed from previously provided ones, this will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.

Parameters:

min - minimum latency

max - maximum latency

setLookahead

public void setLookahead(int num)

Sets encoder lookahead (in units of input rate samples)

Note: This value will be reset to 0 every time before GstAudioEncoderClass.set_format() is called.

Parameters:

num - lookahead

setMarkGranule

public void setMarkGranule(boolean enabled)

Enable or disable encoder granule handling.

MT safe.

Parameters:

enabled - new state

setOutputFormat

public boolean setOutputFormat(Caps caps)

Configure output caps on the srcpad of this AudioEncoder.

Parameters:

caps - GstCaps

Returns:

true on success.

setPerfectTimestamp

public void setPerfectTimestamp(boolean enabled)

Enable or disable encoder perfect output timestamp preference.

MT safe.

Parameters:

enabled - new state

setTolerance

public void setTolerance(ClockTime tolerance)

Configures encoder audio jitter tolerance threshold.

MT safe.

Parameters:

tolerance - new tolerance

close

protected boolean close()

Optional. Called when the element changes to GST_STATE_NULL. Allows closing external resources.

decideAllocation

protected boolean decideAllocation(Query query)

Optional. Setup the allocation parameters for allocating output buffers. The passed in query contains the result of the downstream allocation query. Subclasses should chain up to the parent implementation to invoke the default handler.

flush

protected void flush()

Optional. Instructs subclass to clear any codec caches and discard any pending samples and not yet returned encoded data.

getcaps

protected Caps getcaps(Caps filter)

Optional. Allows for a custom sink getcaps implementation (e.g. for multichannel input specification). If not implemented, default returns gst_audio_encoder_proxy_getcaps applied to sink template caps.

handleFrame

protected FlowReturn handleFrame(Buffer buffer)

Provides input samples (or NULL to clear any remaining data) according to directions as configured by the subclass using the API. Input data ref management is performed by base class, subclass should not care or intervene, and input data is only valid until next call to base class, most notably a call to gst_audio_encoder_finish_frame().

open

protected boolean open()

Optional. Called when the element changes to GST_STATE_READY. Allows opening external resources.

prePush

protected FlowReturn prePush(Buffer[] buffer)

Optional. Called just prior to pushing (encoded data) buffer downstream. Subclass has full discretionary access to buffer, and a not OK flow return will abort downstream pushing.

proposeAllocation

protected boolean proposeAllocation(Query query)

Optional. Propose buffer allocation parameters for upstream elements. Subclasses should chain up to the parent implementation to invoke the default handler.

setFormat

protected boolean setFormat(AudioInfo info)

Notifies subclass of incoming data format. GstAudioInfo contains the format according to provided caps.

sinkEvent

protected boolean sinkEvent(Event event)

Optional. Event handler on the sink pad. Subclasses should chain up to the parent implementation to invoke the default handler.

sinkQuery

protected boolean sinkQuery(Query query)

Optional. Query handler on the sink pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.6

srcEvent

protected boolean srcEvent(Event event)

Optional. Event handler on the src pad. Subclasses should chain up to the parent implementation to invoke the default handler.

srcQuery

protected boolean srcQuery(Query query)

Optional. Query handler on the source pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.6

start

protected boolean start()

Optional. Called when the element starts processing. Allows opening external resources.

stop

protected boolean stop()

Optional. Called when the element stops processing. Allows closing external resources.

transformMeta

protected boolean transformMeta(Buffer outbuf,
 Meta meta,
 Buffer inbuf)

Optional. Transform the metadata on the input buffer to the output buffer. By default this method copies all meta without tags and meta with only the "audio" tag. subclasses can implement this method and return true if the metadata is to be copied. Since: 1.6

builder

public static AudioEncoder.Builder<? extends AudioEncoder.Builder> builder()

A AudioEncoder.Builder object constructs a AudioEncoder with the specified properties. Use the various set...() methods to set properties, and finish construction with AudioEncoder.Builder.build().