LineIterator#

class langchain_aws.llms.sagemaker_endpoint.LineIterator(stream: Any)[source]#

A helper class for parsing the byte stream input.

The output of the model will be in the following format:

b’{“outputs”: [” a”]}

b’{“outputs”: [” challenging”]}

b’{“outputs”: [” problem”]}

While usually each PayloadPart event from the event stream will contain a byte array with a full json, this is not guaranteed and some of the json objects may be split acrossPayloadPart events.

For example:

{‘PayloadPart’: {‘Bytes’: b’{“outputs”: ‘}} {‘PayloadPart’: {‘Bytes’: b’[” problem”]}

‘}}

This class accounts for this by concatenating bytes written via the ‘write’ function and then exposing a method which will return lines (ending with a ‘

‘ character)

within the buffer via the ‘scan_lines’ function. It maintains the position of the last read position to ensure that previous bytes are not exposed again.

For more details see: https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/

Methods

__init__(stream)

Parameters:

stream (Any)

__init__(stream: Any) None[source]#
Parameters:

stream (Any)

Return type:

None