ByteStream, File & Network IO

ByteStream

For input / output of binary data.

Writing

• out.close()
• out.write(Byte[])
• out.flush() writes the data buffer (the ostream user-level cache) to the operating system.
  • This protects against data loss in the event of a program crash.
• out.flushAndSync() calls flush(), then
  • calls fsync() to write the kernel buffers to the file system and then to the hard disk/SSD (the write cache should be written/cleared, too).
  • This protects against data loss in the event of a program or system crash.

Reading

• in.read() -> Byte[] reads
  • everything from the istream user-level cache, if not 0,
    otherwise everything from the kernel buffer/cache:
    • With pipes/sockets this is everything currently in the kernel pipe/socket buffer (typically 64 KB).
      • Blocks when this buffer is empty.
      • When the pipe/socket is closed (and no data is buffered anymore), then it returns an empty array.
    • With files this is everything currently in the kernel “read ahead” cache (typically 64 to 256 KB).
      • Blocks when this cache is empty.
      • When the end of file is reached (and no data is cached anymore), then it returns an empty array.
• in.read(Int n) -> Byte[] reads exactly n bytes.
  • Blocks until the given number of bytes are read.
  • Throws an exception if end of file is reached (or pipe/socket closed) before n bytes are read.
• in.read(Int minimum, maximum) -> Byte[] reads everything that is currently available, up to the given maximum number of bytes.
  • Blocks until (at least) the minimum number of bytes are read (may return immediately with an empty array when minimum is 0).
  • in.read(minimum..maximum) -> Byte[]
• in.readAll() -> Byte[] reads everything until the end of the stream.
  • With pipes/sockets, it blocks until the pipe/socket is closed.
• in.readInto(Span<Byte> buffer, Int minimum = 1) reads into the given buffer.
  • Blocks until (at least) the minimum number of bytes are read (may return immediately with an empty array when minimum is 0).
  • Throws an exception if end of file reached (or pipe/socket closed) before minimum bytes are read.
  • The effective maximum if defined by buffer.size().
    • You may limit the maximum number of bytes to read by using buffer.subspan(0, 4096), or configure the starting point (in the buffer) by using buffer.subspan(100).
  • Usually more efficient, as the buffer is reused and less allocations are necessary.
• in.available() -> Int says how many bytes are immediately available for reading.
  • Returns the size of the istream cache, if not 0,
    otherwise reports the size of the kernel cache/buffer.
  • As that is the number of bytes you would get with the next in.read().
• in.peek(Int n) -> Byte[]
  • Blocks until (at least) the given minimum (n) number of bytes are read.
  • May throw an ArgumentException("Unable to peek() more than ... bytes.").
  • TODO Limited to 16 bytes or to the buffer size?
• in.ignore(Int n) ignores/discards n bytes from the input stream.
• in.ignoreAll() ignores/discards everything that is currently in the input stream.
• in.atEnd() returns True if
  • the end of the file is reached (or the pipe/socket is closed),
  • and no data is buffered anymore (neither in the istream user-level cache, nor in the kernel cache/buffer).

File IO

File, derived from ByteStream.

• file.size() -> Int
• file.position() -> Int
  • file.setPosition(Int n) (AKA file.seekFromStart())
  • A common position for read and write.
• file.seek(Int offsetToCurrentPos)
  • offsetToCurrentPos can be positive (moving towards the end) or negative (moving towards the beginning).
• file.seekFromEnd(Int distanceToEnd)
  • distanceToEnd is 0 or positive (here moving from the end towards the beginning).
• file.truncate() truncates the file at the current position.
  • file.truncateAt(Int n) truncates the file at the given position.
• file.path() -> String

Network & Device IO

Note
This is not necessarily part of the core language. I’m just thinking about what a good API might look like.

• NetworkConnection, derived from ByteStream,
  • a base class for TCP/IP, Bluetooth RFCOMM, infrared, …
  • connection.connect(...)
  • connection.disconnect()
  • connection.isConnected() -> Bool
  • connection.remoteAddress() -> String
  • connection.localAddress() -> String for finding out which interface (WLAN, LAN, VPN) the connection is actually running on.
  • connection.readTimeout() -> Duration
    • connection.setReadTimeout(Duration)
• TcpConnection, derived from NetworkConnection
  • connection.shutdownWrite() sends FIN (half-close), allows further reading.
  • connection.connectionTimeout() -> Duration
    • connection.setConnectionTimeout(Duration)
  • connection.remotePort() -> UInt16
  • connection.localPort() -> UInt16
  • connection.noDelay() -> Bool
    • connection.setNoDelay(Bool disableNagle) to disable the Nagle algorithm.
  • connection.keepAlive() -> Bool
    • connection.setKeepAlive(Bool) prevents connection termination due to inactivity.
  • connection.protocolVersion() -> Int returns 4 or 6.
  • connection.receiveBufferSize() -> Int
    • connection.setReceiveBufferSize(Int bytes)
  • connection.sendBufferSize() -> Int
    • connection.setSendBufferSize(Int bytes)
• LocalConnection, derived from ByteStream
  • connection.path() -> String returns the file system path (for Unix sockets) or the name (for pipes).
  • connection.peerCredentials() -> String returns the process ID (PID) or user ID of the other party.
    • TODO Move to UnixDomainSocket? But on Windows this info is available for pipes, too.
• SerialConnection (RS-232/UART)
  • setBaudRate(Int)
  • setParity(Parity)
  • setDataBits(Int)

Class Hierarchy

• ByteStream
  • File
  • MemoryStream as RAM buffer.
  • NetworkConnection
    • TcpConnection
      • TlsConnection / SslConnection
    • SshConnection
  • LocalConnection for interprocess communication.
    • Pipe
    • UnixDomainConnection in stream configuration.
  • BluetoothRfcommConnection Bluetooth RFCOMM
  • DeviceConnection
    • SerialConnection for RS-232/UART.
    • UsbConnection for USB bulk transfers.
• MessageChannel for message/packet/frame/datagram-based protocols (i.e. not only a stream of bytes).
  • UdpSocket for UDP over IP.
  • UnixDomainSocket in datagram configuration.
  • Communication with sensors on microcontrollers
    • I2CDevice (register read/write cycles)
    • SpiDevice (chip-select-controlled frames)
    • CanBusNode
  • BluetoothL2CapConnection Bluetooth L2CAP
  • ZigbeeEndpoint
  • WebSocketConnection (message frames over TCP)