Use cases · Databricks & Apache Spark

From the shop floor to the lakehouse

What the connection in pronubes brings you.

Machine and process data from OPC UA, MQTT, PLCs and SAP flows into Databricks or your own Spark cluster without additional middleware. Three typical applications at a glance.

AI and ML on current production data

Feed models with data from your plant

Machine and process data is loaded continuously into the lakehouse or your Spark cluster. AI/ML then works on current production data.

Loaded continuouslyData goes straight from pronubes Edge into the lakehouse or your own Spark cluster.
Many sources, one destinationData from OPC UA, MQTT, PLCs and SAP flows ends up in the same place.
One layer lessNo additional integration layer is needed.

Analytics and reporting

Reports on continuously loaded data

Analytics and reporting use production data that arrives continuously in Databricks or Spark. You set up the connection like any database connector.

No SQL knowledge neededSetup works like any database connector, entirely without SQL knowledge.
Own driverpronubes Edge ships its own driver, no Java runtime is required.
Databricks or your own clusterThe target can be Databricks or your own Spark cluster.

Results back towards production

The loop closes

Predictions, setpoints and master data from the lakehouse return in a controlled way towards production through a configured flow.

PredictionsResults from your analyses are available for the way back into production.
Setpoints and master dataSetpoints and master data from the lakehouse take the same route.
Configured flowThe way back runs through a flow that you configure yourself.

Good to know

Plainly said: prerequisites and limits

OperationRuns in a Linux/Docker container and on Windows. The native Windows installation of pronubes Edge is currently a preview.
Update and deleteOnly possible on Delta tables.
Kerberos and data typesKerberos only with the “Binary” transport, complex Spark types arrive as text.
Your next step

Time to get your production data moving

In a short conversation, we show you where these updates give you the most leverage in your system landscape.

Good to know

FAQs

Do I need programming or SQL skills?

No. You set up the connection like any database connector, entirely without SQL knowledge. pronubes Edge uses its own driver, no Java runtime is required.

Which data can I send to Databricks or Spark?

Machine and process data from OPC UA, MQTT, PLCs and SAP flows goes into Databricks or your own Spark cluster without additional middleware.

Can results return to production?

Yes. Predictions, setpoints and master data from the lakehouse return in a controlled way towards production through a configured flow.

Are there limitations?

Yes. The native Windows installation of pronubes Edge is currently a preview, update and delete only work on Delta tables, Kerberos only with the “Binary” transport, and complex Spark types arrive as text.