Sisense partners with Microsoft to maximize nosql performance by running it on iMS's infrastructure-as-a-service platform Azure

SiSense and Microsoft bring big-data analytics to the Windows Azure cloud (by Andrew Brust for Big on Data | January 30, 2013 -- 19:01 GMT (11:01 PST))

Last June, when Microsoft announced a new infrastructure-as-a-service (IaaS) option on itsWindows Azure cloud platform, it became much more competitive with Amazon EC2 as a platform for software-as-a-service (SaaS) solutions. On the NoSQL side, companies like MongoLab andCloudant added Azure data centers as deployment targets right alongside their EC2 counterparts.

(Credit: SiSense/Microsoft)

Today that list of data service providers has grown to include Redwood City, CA and Tel Aviv-based SiSense, and its Prism BI and Big Data Analytics technology. This development has impact on BI, big data, Microsoft, and the cloud in general, and they're worth pointing out.

The offering

SiSense's "ElastiCube" technology is based on a combination columnar database and data-visualization platform.

• In common with in-memory technologies like SAP HANA, Prism uses column-store technology to make short work of data analysis. But since SiSense uses a hybrid disk-and-memory approach, it runs just fine on conventional servers with conventional (and more affordable) amounts of RAM. A memory-only product like HANA needs much beefier infrastructure.
• SiSense Prism also competes with the likes of Tableau and QlikView, which offer data visualization and dashboarding alongside columnar data engines. But QlikView, along with Tableau when used with its built-in engine, can hit limits in scale that SiSense is not vulnerable to. Granted, Tableau can query its data sources directly to avoid such limits, but that can make things more complex, especially in the cloud.
• SiSense licensing is offered on a subscription basis, featuring costs correlated with the number of users. You can install SiSense yourself, on your own servers, on Windows-based Amazon EC2 instances, and now on Windows Azure persistent Virtual Machines (VMs). SiSense will support you in all three scenarios.

SiSense's combination of subscription-based licensing and do-it-yourself (DIY) installation is somewhat unusual. Typically, SaaS-based products put a black box around the infrastructure and give you a Web-based portal to provision your service and get up and running. You can hire SiSense to make things more turn-key for you if you want them to, but the installation is pretty straightforward, especially for personnel comfortable with provisioning VMs in the cloud.

To the cloud, and back

But on the Windows Azure platform, the DIY approach has some interesting ramifications. Windows Azure VMs are based on the same format and technology as its Hyper-V on-premises virtualization platform. This means that you can set up SiSense in an on-prem virtualized environment, then move it to the cloud. Or you could do the reverse. Or, by cloning a VM image, you could do both. Or you could change your mind and migrate between cloud and data center, repeatedly.

The plot thickens. One issue with BI and big-data analytics in the cloud is that getting the data to the cloud can be slow and difficult. But for SiSense customers who are already on the Windows Azure platform, and who are keeping some or all of their data in Azure SQL Database (Or SQL Server on an Azure VM), SiSense can read that data quickly. And if the SiSense VM is provisioned in the same data center as the source database(s), customers should be able to avoid data egress charges as well. (Note that this is my own conclusion, and not a selling point conveyed by SiSense.)

So SiSense offers in-memory analytics, with on-prem-to-cloud and cloud-to-on-prem portability, and because it's not a black box SaaS offering, it can take advantage of data locality and the performance and economic benefits therein.

What would Redmond do?

But what about Microsoft? Its PowerPivot and Analysis Services Tabular mode engines offer column store, in-memory analytics much like SiSense's and its Power View technology offers interactive data visualization and exploration, which is not unlike SiSense's. But Microsoft doesn't offer PowerPivot and Power View as a service on Azure, and yet here it is partnering with SiSense to offer that company's competing offering. What gives?

If we put aside the fact that SiSense's VP of Marketing, Bruno Aziza, was for many years on the BI team at Microsoft, which I think we should, then there are a couple of important conclusions to draw:

• The IaaS push on Windows Azure is all about agnosticism: You can run most major NoSQL databases instead of Azure SQL Database, and you can do so on Linux as well as Windows. You can host applications written in any number of programming languages (including Java, Node.js, Python, and PHP), not just code created in Microsoft's .net development environment. And now you can run competing analytics stacks too. Why not? Windows Azure seeks to be a general-purpose infrastructure environment. To be competitive, it has to do that.
• The Microsoft Data Platform Group, which is an outgrowth of the SQL Server team, is embracing interoperability. Use Microsoft analytics products with non-Microsoft data sources, or use non-Microsoft analytics products with SQL Database and SQL Server as data sources. You'll need SQL Server licenses or SQL Database subscriptions or HDInsight/Hadoop services in any case. It's all good.

Big cloud

What does this mean for the big-data analytics world? First, analytics is going mainstream, and can be OpEx funded through subscription offerings. Second, analytics in the cloud is becoming very practical and can be co-located with transactional databases in the cloud, thus mitigating issues of broadband limits and latency.

Microsoft is still entering the big-data arena. SiSense has fewer than 50 employees. The partnership may seem of small significance. But its ramifications are big.

In-memory vs in-chip analytics

(Re-post) Forget in-memory — SiSense raises $10M for in-chip analytics (by Derrick Harris)

While the rest of the world is agog about big data and in-memory analytics, SiSenseis taking a different tack. It’s rethinking business intelligence with higher-speed analysis on smaller (relatively speaking) data sets by taking advantage of multicore, 64-bit processors. The approach has been paying off with some impressive customer uptake, and on Wednesday SiSense announced a $10 million series B funding round from Battery Ventures along with Opus Capital and Genesis Partners.

Technologically, SiSense is trying to split the difference between just about everyone else doing analytics — expensive full-stack business intelligence vendors such as Oracle, Microsoft, IBM and SAP; big data and data warehouse vendors pushing massive scale databases; and next-generation, visualization-centric vendors such as Tableau and QlikView. It’s fast, it’s has its own columnar database and HTML5 visualization technologies, can scale comfortably up about 100 terabytes, and is designed for business users rather than advanced data analysts.

SiSense’s secret sauce is a processing architecture built for speed even on small machines such as laptops. According to CEO Amit Bendov, the company’s product, called Prism, can handle a terabyte of data on a machine with 8GB of RAM because it relies primarily on disk for storage. Data is only moved to RAM as necessary, and then Prism uses vectorization and optimized instructions (that do one thing only, but do it across all the data that fit the query) to handle as much work as possible in parallel on the processor.

Source: SiSense

“We say in-memory is not the future, it’s the past,” said Bendov. ”We’re already two steps ahead.” Using Hadoop or Teradata for a handful of terabytes, he added, is overkill, “like driving a Humvee to the grocery store.”

Eldad Farkash, SiSense’s co-founder and CTO, uses a different analogy — that of buying beer — to explain the technology’s underlying rationale. Latency to the CPU from the processor’s L1 cache is like grabbing a beer from the refrigerator, whereas using the L2 or L3 cache is like riding a bicycle to the corner store. RAM is the equivalent of driving a car to the grocery store, and accessing data from disk is like going to the brewery itself. Prism knows it will have to go to the grocery store, but it gets as much beer as possible from the fridge and corner store first.

Once users are actually in the product and analyzing data, it’s a drag-and-drop experience to connect various data sources and points (although custom SQL is allowed, too). The actual analysis window features a canvas that can display numerous widgets (e.g. pivot tables, charts or dashboards) at once.

Bendov said SiSense’s revenue grew 520 percent in 2012 and its notable customers include Target, Merck, Samsung and Cisco. The new investment will be used primarily to bolster the company’s sales and marketing efforts — which thus far have been largely relegated to in-bound inquiries — and to support customers in different geographies (the company is based in Redwood Shores, Calif.). “Now’s the time to add oil to the fire,” he explained.

As impressive as it all sounds, though, SiSense’s biggest challenge might well be getting noticed above the fray that is the analytics space right now — especially among more well-known and arguably future-proof vendors and technologies. That said, being a low-cost option that users like and that actually works has proven remarkably effective in an era of cloud computing and bring-your-own-device, and SiSense appears to racking up users at a pretty rapid clip.

Any product that can prove its worth initially with the people who have to use it stands a good chance of sticking around and becoming a permanent part of IT budgets for years.

Feature image courtesy of Shutterstock user Iscatel.

-----------  Interesting comments by the CEO of SISENSE to reader's response--------------------

Rob Klopp

"There are some marketing semantics here… The amount of cache on a Xeon processor is small… 30MB of L3, 2.5MB of L2, and 256K of L1 on the highest end Xeon E7-8870 (see http://ark.intel.com/products/53580).

Since a single analytics user drinks gallons of beer per second… and 100 users drink hundreds of gallons per second… the refrigerator is hardly adequate as is the bicycle… and you end up driving all of the time… in other words Sisense is really an in-memory DBMS (after all… every DBMS and every program ultimately gets data from the L1 cache/refrigerator every time).

Further, you need to go to the brewery for 100TB every time… and you need a big truck or lots of trucks running in-parallel to keep up. So the idea that this is in-chip and comfortably supporting 100TB is hard to rationalize unless Sisense is also a multi-node, shared-nothing, scale-out DBMS.

Finally, HANA works hard to load data into cache in full cache lines and to use SIMD instructions (vectorized and optimized)… so Sisense feels like HANA R01. This is very cool… but the “Forget in-memory… in-cache” headline is pure marketing."

Eldad

"Hi Rob,

since you used a technical hypothesis on a marketing article, I’ll try to give some details on why “in-chip” is precisely the term to describe what we do:

With “in-chip”, we mean 4 things:

1. Having a query kernel that is both vectorized & cache-aware
2. Having a JIT compiler that converts SQL into c/asm code
3. decompress data in-cache to save memory bandwidth and avoid roundtrips between RAM & CPU
4. apply columnar format not just for storage, but for re-using intermediates between query operators

finally, a few comments of my own:

1. There is nothing “hard” about loading “full cache lines”.
You always load full “cache lines” into the cache, thats just how CPU caches work

2. SIMD is trivial, its cache-awareness thats complex
The way you slice arrays into vectors inside the CPU cache is extremely complex because it must happen in real time, its based on the actual query expression, the available memory bandwidth, and the instruction set version of your CPU.

3. You can use SIMD intrinsics as much as you like, but you’ll still get a lousy performance out of your in-memory database because of the excessive branching and software abstractions that db’s are so hooked on. If your CPU can’t predict, it can’t prefetch, and your “fridge” will be empty most of the time, waiting for the trucks to arrive with “yet another single bottle of beer”.

4. SiSense is in-memory because it utilizes the RAM as much as it can, but it also uses compression & vertical fragmentation to keep ALL data on disk. Unlike most databases, we don’t use a page buffer but rather memory mapping. Aside from the raw performance in streaming data from disk, for us, keeping the CPU on actual query operators instead of buffer management is crucial.

5. Every program gets data from L1 cache, but most of them are not cache aware. The difference between having the data in cache when you need it, and waiting for it to be fetched from RAM is x50-x100 times slower.
the beer analogy was used to describe exactly that…

I hope this gave you some more details into the abstraction we used to explain the technology.

Cheers,
Eldad
- “Software abstraction is beautiful, but there’s nothing beautiful about slow software” "

BI Market Size by 2016 (users & revenue) + key players

QLIK, Tableau: New Generation of Analytics Software Can Expand Market (source:Barrons)

Pacific Crest‘s Jesse Hulsing today reflects on the new breed of corporate “business intelligence” software created by the likes of Qlik Technologies (QLIK), Tibco (TIBX), and privately held Tableau Software, which he thinks is “hitting critical mass” in a $14 billion market.

As Tableau IPOs, whenever that arrives, it will not only expand the BI market, writes Hulsing, but also the 5% share of the market that the new players maintain, with about $800 million in annual revenue. Tableau had about $127 million in revenue last year, writes Hulsing, as opposed to Qlik’s $389 million.

Qlik’s and Tableau’s share can double by 2016, he thinks. On top of that “only 17% of information workers use data dashboard or BI tools as part of their job,” he writes, citing data from Gartner, IDC and Forrester. There is probably a market of 60 million people in the developed and developing worlds who could be users of such software, thinks Hulsing, versus the 13 million to 14 million who currently are addressed by the available wares.

The “old paradigm,” in Hulsing’s view, is that brought about by SAP AG (SAP), Oracle (ORCL),International Business Machines (IBM), and past stars such as MicroStrategy andInformation Builders. New stars include not only Qlik and Tableau and Tibco’s “Spotfire” product, but also Prognoz, Targit. And there are even “emerging growth opportunities” inRoambi and Yellowfin (for mobile); Platfora, Datameer, Karmasphere, Alteryx (Big Data);Domo, GoodData, Birst (software-as-a-service); Jaspersoft, Pentaho (open-source software).

Among the things helping Qlik and Tableau increase sales faster than the overall market is the sense among users that most “are unable to access the information they need in a timely fashion,” writes Hulsing, citing survey data from DOMO. Obstacles with the older software include “canned reports,” “lack of APIs and openness,” “heavy upfront implementation time,” he writes. The most prevalent method today of sharing insights is disseminating spreadsheets built in Microsoft‘s (MSFT) Excel, notes Hulsing.

The Qlik and Tableau tools have “lower price points, lower implementation costs” and are easy to use, he writes. As evidence of broadening appeal, Hulsing notes that Qlik has 27,000 customers as it approaches $400 million in revenue, whereas tools sold by Cognos reached only 19,000 customers at a similar revenue stage, and Business Objects had only 15,000 customers at a similar point in its growth.