Streaming

Reactive Streams implementation with back-pressure: Source, Flow, Sink, graph DSL; Alpakka connectors (Kafka, S3, JDBC...).

35.1 fs2 (functional streams for Scala)

//> using dep co.fs2::fs2-io:3.11.0
import cats.effect.*
import fs2.*
import fs2.io.file.{Files, Path}

val wordCount: IO[Map[String, Int]] =
  Files[IO].readUtf8Lines(Path("big.txt"))
    .flatMap(line => Stream.emits(line.split("\\s+").toSeq))
    .filter(_.nonEmpty)
    .fold(Map.empty[String, Int])((m, w) => m.updated(w, m.getOrElse(w, 0) + 1))
    .compile.lastOrError

Stream.range(1, 10).evalMap(i => IO.println(i)).compile.drain
Stream.awakeEvery[IO](1.second).take(3)
stream.parEvalMap(8)(process)            // bounded parallelism
stream.chunkN(100).evalMap(batchInsert)

35.2 ZIO Streams

ZStream.fromIterable(1 to 10).mapZIOPar(4)(i => ZIO.succeed(i * 2)).runCollect

35.3 Akka/Pekko Streams

Reactive Streams implementation with back-pressure: Source, Flow, Sink, graph DSL; Alpakka connectors (Kafka, S3, JDBC...).