Custom ETL - A Product Or A Service?

Ethan
CEO, Portable

Are custom ETL pipelines a product or a service? It really depends on the development methodology and reusability.


At Portable, we build custom ETL connectors on-demand for clients.

Why?

Because there are so many business applications in today's enterprises.

And other ELT tools only support a subset of the universe.

When a data team wants to build a dashboard or automate a workflow through their data warehouse, they need a simple way to get data from their business applications into their data warehouse.

If you can't find an off-the-shelf connector, you need to find someone that can build the integration. That's where we help.

This sounds amazing, but are custom ETL connectors a product or a service?

People always ask. Are custom ETL pipelines a product or a service?

To answer the question, let's consider 4 aspects of the custom ETL lifecycle.

  1. Development
  2. User Experience
  3. Reusability
  4. Maintenance

Development.

Every data integration - whether it's a product or a service - has to be developed. The easiest way to differentiate a product from a service during the development phase is whether the end user has to pay for development.

In many scenarios, services teams or consultants will charge an up front amount for development of a custom ETL connector. In that scenario, it's pretty clear that you are paying for custom development services instead of a SaaS product.

At Portable, we do not charge clients for development of custom ETL connectors from APIs into data warehouses.

Why?

Two reasons:

  1. When we build custom ETL connector, it automatically becomes a templated part of our broader catalog.

  2. We've spent years developing a platform on which we can build new, custom ETL connectors on-demand for clients, unbelievably efficiently.

User Experience.

The second consideration when determining whether a custom ETL connector is a product or a service is how you - as the end user - will configure and set up the data flow.

If you have to deploy code or infrastructure, it's probably a service.

If your vendor needs to set things up behind the scenes, it's probably a service.

At Portable, our clients can sign up and log into our product directly. We build connectors on-demand for clients (for their particular custom ETL needs), but once the connector is live, it shows up in our product alongside the 200+ other connectors we offer. It's available to every prospect and client.

Once a connector goes live, it has monitoring, alerting, instructions, and email notifications out of the box. We pride ourselves on the seamless product experience we provide clients.

Reusability.

If the partner or vendor you work with is building a special snowflake workflow for you, and therefore expects you to be the only user of that particular custom ETL connector, you should be worried.

They won't be able to amortize the cost across clients.

When they encounter issues (which they will), they'll think about your custom ETL pipeline as a one-off workflow instead of as part of their core product.

At Portable, each connector becomes a template. Once a single client goes live moving data, we aim to get many more clients moving data through the same pipeline as quickly as possible.

On our side, we don't have to do any additional technical work to get 100 clients live on a particular custom ETL pipeline vs. 1 client.

We expect to reuse every connector across many clients.

The first user might receive a customized experience, and ask 'is this a product or a service'.

The second client receives a purely product-oriented experience for the exact same connector.

Maintenance.

If a vendor has to maintain each connector in a one-off capacity, or if they have special monitoring to add during development, it's likely a service.

You don't want your critical business data flowing through pipes that aren't meant to be secure, stable, and reliable.

At Portable, when we ship a new connector, it automatically ties into our production-grade monitoring, alerting, distributed systems scalability, and automatic rate limiting out-of-the-box.

We apply the same centralized approach to every custom ETL connector we build.

Each custom ETL connector might seem custom to you.

At Portable, they are just part of our product!


Want to learn more? Book time for a discussion or a demo directly on my calendar