Zapier Usage Research

In July of 2013 I did some business related research regarding the type of usage made in Zapier. The research involved scraping the list of zaps that’s publicly available on the Zapier website, dumping them into a database, and fetching a few queries with different intersections on the dataset. The business I was planning at the time is now acient history, so I thought I’d share my results with the world, in case anyone might find it useful.

Patents Are Going To Change

Within about two months time from the publication of this post, the patent for the portable computer is going to expire. Considering the proliferation of laptops, and the amount of patents involved with the manufacture of one, it’s hard to predict whether this change is going to affect the industry to any extent. However, this does illuminate clearly the point in time we’re in within these early stages of the age of information. It’s well known that the patentability of software was established within the U.S during the early 90’s, and that they’ve been growing in numbers ever since. This is important for us, because these days the first few batches of computer related patents are being released for free use. It is the unique character of this industry that makes this advancement un-newsworthy, because of a perceived pace of progress under which nothing of 20 years of age could be of any worth. Anyone who’s been around that long, though, knows this to be untrue. Innovation works much the same as a pendulum, moving from one extreme to the other. Such is the way Anil Dash describes the move from mainframes to the personal computer as a revolution of sorts, only to be followed by a swing of the pendulum in the opposite direction with the proliferation of the cloud. The same could be claimed with regard to IDEs, visual programming languages, imperative programming paradigms, and many other aspects of technology that we consider as innovative. Today’s world of technology is without a doubt on the move forward, but much of the conceptual foundations of today’s inventions were already laid out in legal speak in a previous turn of the wheel. This has the dual effect of allowing us to move forward in fronts that were blocked from extensive (as opposed to intensive) progress up to now, and also, limiting the possibility of a future appropriation of these innovations.

The Opportunity

When speaking of an extensive progress that weren’t possible up to now, one can imagine a world of smaller and smaller manufacturers, much like the way every town had it’s own bread makers and shoe makers at some point in time. This might seem counter-intuitive with regard to a common rationale of centralized manufacturing that can cuts costs, but this isn’t really the case in most industries. Nike has centralized manufacturing, but their products cost a fortune, because they have vast expenses for the purpose of brand building, and because there’s enough of a demand to make it worthwhile. The telecommunications equipment industry is another kind of example. A few manufacturers like Cisco have vast expenses in R&D and they aim for the higher end of the market. However, most of the consumers are service providers in developing nations, where infrastructure has yet to be deployed. In these markets there seems to be a clear preference for smaller manufacturers that provide cheaper goods. Another result of the same process could be better kinds of free software alternatives to existing products, and in particular a preference for maker alternatives or pretail variations that could replace the mass-manufactured goods of today’s world. The rise of 3D printing and maker culture saw an instantaneous reaction from IP aficionados that claimed for the need to implement IP protection mechanisms within printers, even for non-commercial use. But the expiration of patents for much of the technologies that are simple enough to be of relevance to makers would change the balance of power in these regards.

The Future Of Patents

Today’s innovative sphere is advanced in two parallel trajectories. On the one hand are entrepreneurs, hackers and makers, and on the other, a narrow oligopoly of tech giants. In a recent talk with Israel Twito, CEO of New-tone Patent Search ltd., it was noted that the same balance of power between big manufacturers that prevents smaller players from being able to compete, has the double-sided effect of forcing these same companies to continue the arms race forward, with intensive R&D, as well as through the constant acquisition of smaller players. When passing criticism over an exits-focused startup culture, one has to keep this state of affairs in mind. Some industries are blocked from entry to anyone that isn’t deep-pocketed enough to join into an arms race that’s already under way. On the other side of the market are big players that need sources of producing new IP, with little or no regard to the cost – both the financial one, and the one that has to do with the damage to innovation and to clients of acquired ventures. This saddening state has an upside as well, one that will only begin to show within the next few years. The tech giants are arming themselves with patents for their own ends, but are also making the patent system obsolete by way of doing so. With the realization that a patent’s strength lies in the abstractedness and generalization of its phrasing, and with the constant swings of the technological pendulum, we’ll see more and more areas where the field for innovation is ironically already covered by existing, expired patents. With that process underway, the tech giants of today will be forced to create products of such sophistication that their target audiences will narrow, much like Cisco’s does today.

An End Note

The grinding of the patent system by economically motivated and heavily funded players requires new and better tools to help us understand what opportunities lie before us. A survey of U.S patents that are about to expire within the next year shows interesting things like cloaking systems, software protection machanisms, electric car chargers, pen-based computer inputs, home networking, and a lot of the technologies around cellular communications. Crossing that information with patent lawsuit databases shows that not a lot of today’s expiring patents are inhibitors to innovation (or competition), but that’s going to change at some point in time, and we best be prepared.

My Cofounder Video Marketing Endeavors

I prepared a short wanted ad for my next venture with the kind help of my beloved partner. It got me to the first page of Youtube’s search results for the keyword “find a cofounder” with just friends and family marketing. It didn’t get me a cofounder. This indicates some relatively obvious conclusions: (1) There’s ad arbitrage on Youtube, and (2) this doesn’t matter if your target audience isn’t looking for you over there. Either way, here it is:

PinMyScreen – Bookmarks Using Pinterest

PinMyScreen is a Social bookmarking extension for Google Chrome. The extension adds a Pinterest pin button to the browser’s toolbar, and allows the user to pin website screenshots onto his Pinterest boards. This works the same way as with gimme bar only it doesn’t impose a new system on clients. Instead, PinMyScreen allows you to use your existing Pinterest acocunts to save your favorite websites.

Known Issues

PinMyScreen uses Chrome’s built-in screenshot functionality (the same one that’s used for the ‘most visited’ page you see when opening a new tab. This, for security reasons, prevents us from screenshoting any sensitive data that might exist within secured pages or Chrome web applications. As a result, not every website can be shot, and you’ll sometimes find that the extension sends an empty image onto the Pinterest ‘pin this’ form. The process of screenshoting a webpage requires a roundtrip request and response with our server. You should note that we do not keep any images or any other sort of information on those servers or anywhere else. As opposed to many other Chrome extensions, I’ve chosen not to use anonymous analytics as well, so no data on your actions is logged at any stage. However, communication with our servers sometimes takes a bit longer than expected. You should note, that if you clicked the ‘pin this’ button on your browser’s toolbar, the request was sent and the pinterest form will appear in a few seconds time. This however doesn’t always happen fast enough, and there isn’t an indication that the request is in a processing state. If I’ll hear a lot of complaints from people about speed, I’ll transfer the servers onto Amazon’s cloud services, so if you think that there’s place for improvement, just let me know.

Bug Reporting and Support

I’m always available to hearing feedback over the extension, be it a positive or negative one, so don’t hesitate. If there’s anything you need help with, just send me an email to me at shayacrich dot com.   Visit our page on the Chrome WebStore to add the extension to your browsers.

Developer & Vendor Tools for Data Integration

Imagine a scenario where you launch a new project management tool, and it’s really the best one out there, but no one wants to use it. This could happen for several reasons, one of which would be the lack of integrations with other types of software your potential users are using. They might need integration with Google Calendar, or with a variety of CRMs, or ticketing systems, or bug tracking systems, or version control services, and so on and so forth. Eventually, you’re forced to start providing built-in integrations with any and every other tool out there. With each new integration offered, the potential user base of your app grows bigger, which is good, but this scenario has it’s disadvantages:

  • There are infinite potential integrations, so the work is unending.
  • This diverts the attention of your development staff from what they do well (project management, in the case described above) to other types of work.
  • Bugs are usually hard to control because you’re working against systems that you don’t host, and whose code you can’t see.

The need for integration tools aimed at software vendors derives from the scenario described just now, which I like to call the integration pitfall. Application providers are faced with pressure, having to offer built-in integrations with a growing subset of related products, so that their own product would be useful to new target audiences. Some tools were developed with the aim of assisting these software providers, instead of targeting their end users directly.

Elastic.io is one such tool. It lets you connect with a variety of third party services and map fields from these data sources to your own application with a simple drag and drop interface. In sum, Elastic.io is meant for non-technical users that want to extend the application that they offer with built-in integrations. A very different approach is that of Nectil, a French service provider that developed their own software for web application integrations, and that offer small businesses tailored solutions on top of that framework.

Most other tools aim at the developer crowd, trying to help them consume APIs with greater ease and speed. Temboo provides SDKs for server side API consumption. It supports a very extensive amount of APIs and provides access to them through a unified interface. The service is SaaS based, though cheap, and meant for low levels of consumption (anything really big would require custom pricing). Webshell is an API that lets you integrate different APIs from third party services and create new functionality. It’s a JS library, which handles authentication for you and lets you construct your custom APIs using an editor. DERI Pipes is a software that lets you create automated processes that transform and mash together web content (much like Yahoo! Pipes, but more technically complex).

Cumula is a PHP application framework that takes the idea even further. Based on the assumption that modern web applications no longer sit on top of a single on-premise database, but instead connect with a multiplicity of services, they’ve contrived a framework that lets developers build applications with a unified interface for local and remote data sources. It’s modular, with the use of components for describing parts of the application. Each component has its own routing and templating. A collection of DataStore and DataService packages are available as dependencies, included in the same Github repo. As of now, Github’s popularity metrics indicate very low acceptance rates, and the same goes for the associated Google Group. Nevertheless, this is a relatively new project, and they’ve already received some media coverage.

Other platforms aim to help developers find and consume APIs, without altering the way they work or requiring the utilization of a specific framework. APIHub (renamed to the anypoint portal after being acquired by Mulesoft) and the ProgrammableWeb projects are API explorers, within which one can quickly search and learn about new APIs. Mashape does the same, although focused more on developer APIs and less on connecting with third party applications. Mashape acts as a middleman between providers and consumers, relieving API consumers from having to maintain separate authentication processes with a multiplicity of remote services, and thus enabling the proliferation of usage of such services.

Edit: Since the time of writing this, a new contestant has emerged, called CloudElements, that provides a uniform API for accessing multiple cloud based applications.

Overview of Process Automation & BPM Solutions

Process automation tools, meant for either small businesses and consumers, or for Enterprises, allow non-programmers to combine services from different vendors into a new piece of functionality. Enterprise and consumer tools differ from one another in the level of complexity and technical know-how that they require. Most tools meant for consumers can be configured with no more than a few simple steps. They allow people that don’t write code to define automated behaviors between cloud-based services, such as sending yourself an email when this or that happens, adding a new contact to your mailing list or CRM when a form is submitted somewhere, or sending new content directly to your Dropbox folder. These can be regarded as productivity micro-tools, and are very limited in the scope of possibilities that they offer.

Zapier (YCombinator alumni) is a prominent tool, meant for very small businesses and consumers, which offers an extensive list of third party services to integrate with. IFTTT (a Betaworks profile company) is another prominent contestant, meant mostly for consumers and free of charge. Cloudwork (sister company of GetApp) covers the same space as well, and are working on a set of SDKs, and a visual editor, to allow anyone to create new integration types. Yahoo! Pipes is a tool to aggregate, manipulate and mash up feeds from third party services, that offers lots of useful functions for processing the data and is all assembled visually. Other alternatives in this sphere include RunMyProcess, Foxweave & WeWiredWeb.

BPM software for Enterprise customers are the other end of the spectrum. They normally provide form designers, visual process editors, rule engines and customizable dashboards for monitoring workflows. These solutions are relatively complex to operate, and usually require a learning period to adapt to. Their pricing levels are clearly meant for the Enterprise world. Competitors in this sphere include Appian, Skelta, Rollbase, Iceberg and ProcessMaker, to name a few.

Overview of Dashboard Solutions for Data Integration

The tools surveyed here offer an integration of data into a single platform, but do so only in one direction. By this, I mean that these solutions don’t offer the possibility of sending updates back to the applications from which the data originally came from. One such tool is Sumall, which offers a marketing-focused set of metrics based on third party data, consumed via APIs. A competitor in the same vector is Geckoboard, offering an API itself for end users to add their own services on top of those provided out of the box.

A slightly more configurable tool is a Chartio, which connects with what are mostly DBaaS providers, to allow one to create his own customizable business dashboards. Chartio’s platform allows non-technical end users to author their own graphs by dragging and dropping fields from the third party data sources they’re connected with. Combined with that, is a rules engine, based on a set of dropdown menus, for setting up the logic behind the presented graphs. A more technical option is that presented by Dashku, offering a platform on top of which frontend developers can build widgets that connect with third party data, and that combined, are gathered into a fully self-authored dashboard.

Every BI software provider offers some level of dashboard building capabilities. Notwithstanding, these solutions are usually out of reach for small businesses. Jaspersoft, for instance, offers a community edition of its solution as an open source project. However, that open source version lacks the dashboard capabilities that paying users are offered.

A subset of dashboard providers is that of analytics solutions. These are less likely to integrate data from different sources and are therefore of relatively minor significance for this overview. In spite of that, some tools are still worth noting. The first of which is Chartbeat, which gathers data from the users’ websites, but integrates with social media data as well, to generate a more insightful perspective, all without requiring the end users themselves to do the heavy lifting data work. Another tool takes that approach slightly further – ThinkUp pulls together data from users’ social networks, but doesn’t offer graphs and dashboards at all. Instead of pouring immense amounts of data onto the screen, ThinkUp presents a feed, much like that of most social networks, that highlights actionable insights based on the sum of the data that was gathered.

Overview of Data Integration Tools

Integration for SMBs

OneSaas is a data integration application that automates common tasks for specific business users, mostly in eCommerce and online marketing. It features a variety of integrated services and a simple interface to connect to these third party user accounts. Then, there are a few options for each type of data, mainly, whether or not to add and delete items according to changes in other systems that handle the same data type – Magento with eBay, Google Contacts with a CRM, etc. It also has an interface that displays all the data from the third party vendors in tables.

Cazoomi SyncApps is an integration cloud meant for small businesses. It features a visual studio for composing integrations, much like Enterprise solutions do, but are aimed at SMBs, both with regard to pricing and available integrations. Cazoomi’s website reports them having 10,000 customers, and a team of 13 members (at the time of writing). It appears that Cazoomi’s solution is meant more towards application vendors that want to create integrations than it is for end users themselves. Another type of tools are web page scrapers meant to ease extraction of data from websites without the use of APIs that sometimes aren’t available. The prominent tool in this respect is Dapper, and a newer contestant is import.io.

UPDATE: Since this text was written, a new service came out that scrapes websites automatically, and let’s you build apps out of the data it extracts, called Kimono

Industry Specific Integration Tools

Each industry has an array of specific tools, either meant for integrating data, or that integrate data as a byproduct of their main function (dashboards, process automation, etc.). These solutions are thus somewhat parallel in their function to data integration software, and could suffice in any scenario in which integration needs are within this specific set of third party applications. However, a common scenario among small businesses is an overdependence on one such specific tool, and an inability to make that tool work with services meant for other aspects of running a business. For instance, a business in eCommerce might utilize an application that optimizes and automates some of the aspects of selling online. Such a tool would surely connect seamlessly with any and every marketplace, comparison shopping engine, shopping cart software, payment gateway and shipping service. However, the ability to make this tool work with a newsletter management system, or a PPC ad platform, is of limited capacity.

Examples of such niche products vary widely from one industry to another. Ad management tools include Datorama, Kenshoo and IgnitionOne. In eCommerce, common tools include Channeladvisor, eSellerPro, Vendio and Auctiva. mHealth apps can be connected with Validic. Data from different analytics solutions can be pulled together with Segment.io. Social media campaigns can be managed from a single place with Hootsuite.

Enterprise Integration Tools

The Enterprise world covers a very wide array of solutions at this point in time. As part of this overview, only SaaS products meant at Enterprise customers will be examined and not hybrid and on-premise solutions. A key character of the following solutions is that they’re meant for IT departments to deploy, and not for the end users themselves.

Informatica Cloud is a high end service for Enterprise clients that offers a large set of tools for integrations and data processing. Jitterbit offers an open source platform for the creation and management of integrations, both legacy and SOA based. They make use of reusable templates for common integration needs so that developers have less work. Extol offers a design tool for building integrations between Enterprise applications. JackBe Presto is a visualization and BI tool that connects to external sources – data warehouses, spreadsheets, feeds, existing BI systems, etc., using a graphical assembly tool. SnapLogic is an Enterprise-grade solution for data integration that makes use of modular pieces of connectivity or functionality called ‘snaps’. Snaps can connect to third party services, local files or anything else, and can provide atomic bits of functionality used for ETL tasks. They have a marketplace for snaps with high end pricing. Mulesoft offers a set of products meant for data integration. Their visual editing tools are meant to ease the work of developers, not to provide a EUP solution. It offers a variety of pre-built connectors and integration apps, the ability to develop new ones, and visual tools for integrating services and for building data transformation processes. Dell Boomi is an Enterprise API integration tool that uses a flowchart interface and offers both pre-built integration types as well as the ability to create new ones or have Dell do the customization for the end users. Scribesoft offers the same standard set of features, as well as the ability to connect SaaS products with on premise software. Other solutions in this sphere include IBM Infosphere DataStage, IBM CastIron, and Talend’s Integration Suite.

How to Serve Cloudinary with HTTPS on Django

Cloudinary’s documentation states that their client side libraries can automatically detect the protocol used in a page and generate HTTP/S urls for the image tags accordingly. This, of course, doesn’t apply to server side code, for which there’s no apt documentation on this matter.

In Cloudinary’s Django library, there are three ways in which you can define your site specific parameters:

  • A dictionary called CLOUDINARY that you can set in your settings.py.
  • The CLOUDINARY_CLOUD_NAME environment variable.
  • The CLOUDINARY_URL environment variable.

Although not mentioned in the documentation, the CLOUDINARY_URL environment variable can take query paramters, and these would be added to the cloudinary object that generates the urls within the Django templates. So to make sure that all requests to Cloudinary are made with HTTPS, all you have to do is set the CLOUDINARY_URL variable with the query parameter ‘secure’ and the value ‘True’, like so:

export CLOUDINARY_URL=cloudinary://123456789012345:abcdefghijklmnop-qrstuvwxyz@abcdefgh?secure=True

Just make sure to replace the fake API key, API secret and cloud name I typed above with your own.

This approach is not so great, though. It forces us to make sure we include the extra query param if we’ll ever need to change the Cloudinary URL in the future, and because it’s not in the code, we can only have all our pages transferred securely or none at all.

An alternative would be to use the CLOUDINARY dict that we can set under settings.py, but that method has a disadvantage as well – it doesn’t play nicely with the other environment variables. In fact, Cloudinary’s Django library regards the three approaches specified above for entering site preferences as exclusionary, so if we set the CLOUDINARY dict under settings.py, the CLOUDINARY_URL will not even be looked at. Considering the fact that under platforms such as Heroku, an environment variable is the preferable route, we’re basically left with nothing but bad choices.

Luckily, the cloudinary object used to generate image links is just python code, the intantiation method of which takes keyword arguments that would be added to the same dict used for storing the site specific values from the inputs mentioned above. So to set Cloudinary to use secure URLs, all that’s required are the following two lines in your settings.py:


import cloudinary
cloudinary.config(secure=True)

That’s it! Your Cloudinary URLs will now always be loaded with HTTPS. Because this is just python code, you can alter it to use secure or unsecure URLs according to different views (just make sure to do that inside view functions and not in settings.py).

Overview of Online Databases & Business App Solutions

These platforms are meant to allow non-technical end users to define data structures, according to their specific needs, and then make use of an online database that stores data according to those definitions. They usually also allow one to build business apps, which are small productivity apps for web or mobile, meant to ease tasks such as data entry and reporting.

The challenge they all face is the technical complexities involved with data modeling, which in this case is left to non-technical end users. The traditional way of solving this is the route taken by MS Access, that is, leaving it to the end user to plan and manually build table schemas and relations. Once the user completes this process, the system can automatically create forms for data entry, based on the schemas he supplied. A more modern approach to solve the same problem goes the exact opposite path, by allowing the user to build the form itself, with a drag and drop interface, then automatically generating a matching schema behind the scenes.

In general, there was a surge of app authoring tools in ’06-’07, and some of those attempts to empower non-programmers to build tools for and by themselves were halted by the ‘08 crisis. Under these circumstances, DabbleDB was acquired by Twitter in ‘10 and shut down in ‘11, Lazybase disappeared in thin air, Teqlo got shut down, and Coghead became part of SAP’s River platform.

The main tool used today for authoring small business apps is Caspio Bridge. It allows one to create databases, web forms, and apps, all without writing any code. With regard to the UI, it works similarly to MS Excel and Access, but is online, with integration support through a SOAP API, POST requests, Datahub, a JS library and a plugin for MS office. Another common solution is Intuit’s QuickBase, which allows the same standard app creation, as well as managing data through a spreadsheet UI. They differentiate by offering a large set of database templates for common industry-specific needs. Both these tools have pricing levels meant for SMBs (but not micro businesses), meaning their plans start at $250-300/mo. Zoho Creator is a third alternative with lower pricing (for small usage, whereas a high usage would cost a lot more than what’s offered by other services). It’s an online database and business apps service, with a form creation wizard, business rules engine, customizable reports, multiple view types (including an editable grid), alerting, rebranding and styles, collaboration, backups, APIs and data exports.

Other contestants in this field would be TrackVia, LongJump, FileMaker Pro and Viravis. They all offer database and app creators meant for non-programmers, as well as custom reporting and database templates. LongJump provides an SDK meant for adding functionality to the system by code. FileMaker Pro offers the ability to build apps for tablets and mobile. Eccentex is another solution, that provides a platform on top of which developers can build productivity apps, and end users can make use of them.

A newer generation of tools is emerging for the past two years or so. Apart from Dreamface, which is aimed at IBM BPM software customers, all of these solutions are meant for micro to small businesses as well as the early adopter consumer market, and are priced accordingly. Ragic! Builder is an app authoring tool that offers a spreadsheet UI meant to allow the end user to define the applications’ form fields. It then offers reporting, full text search, a query builder, embedding results onto one’s website, importing and exporting, access control, versioning, customized scripting and an API. Knack is another solution with really easy data management, user management, and that’s customizable with an open API as well as with CSS and JS editors. SodaDB is a donationware product that offers a simple and customizable database, importing and exporting, a form builder, full text search, and the ability to work without signing in.

Another brand of solutions is that of online spreadsheets, which sit at a crossroad between accounting software and online databases. These are meant to compete with MS Excel on specific vectors, and offer features that make them more attractive for these specific target audiences. Smartsheet is a paid-only online spreadsheet, with a modern UI, mobile versions, and an emphasis on collaboration. Zoho Sheet is a MS Excel clone that mostly fits existing customers of Zoho. Glide Crunch is a desktop spreadsheet application meant for large spreadsheets (the kind that won’t fit into an online tool), and that syncs automatically with your storage on Glide’s cloud office suite. AirXcell is a web based spreadsheet tool meant for scientific and statistical heavy-load work. It uses R language syntax for formulas and functions, and comes built in with a few financial applications. Flextory is a web application that acts as a sort of administration panel for your own data. It’s looks and acts like a standard web admin panel, with item editing forms, filtering, sorting and choosing table fields to view, but the data is really the kind of data you’d use MS Excel to manage. SecureSheet is an online spreadsheet that offers collaboration and security. BinaryThumb is a spreadsheet for iPhones and iPads that contains not only numbers but any kind of media (slightly pivoted since this text was written). Sumwise is a spreadsheet solution that offers smart features like grouping, reusable cells, etc. CollateBox offers collaboration on lists based on MS Excel data. Other solutions would include Editgrid and Skysheet.

UPDATE: Since the time of writing this text in 2012, a new online database solution emerged in the scene, called Team Desk.