IBM

Doors is bought through normal IBM channels. Doors NG is supplied with Doors, if you buy the latest Doors release. A trial version is available.

Doors is targeted particularly at (but not limited to) Systems Engineering professionals, working on high-value, regulated or safety-critical systems; defense, aerospace, health, transport and so on

Doors is a client/server application with its own database and its back-end runs on a range of Windows. UNIX and Linux servers.

IBM offers all the services, available across the globe, that one might expect to need in support of this product.

Particularly noteworthy is the OSLC community. OSLC or Open Services for Lifecycle  Collaboration, is a set of interface specifications which allows Doors to produce/consume services with, say, Rational Team Concert (an Agile application lifecycle management (ALM) solution) or, potentially, other tools supporting OSLC. 

Doors is a market leader in the formal requirements management space, so (allowing for the fact that not every developer is skilled ior experienced with using requirements management), it should not be hard to find Doors expertise. IBM supplies Door consultancy and training courses, both in the classroom and online (video); and a range of external trainers and consultants are also available to facilitate deployment of Doors in the organisation (although always remember that institutionalising reequirements management is as much a cultural/people issue as a technology one).

Informix has been focused on the VAR and ISV market by IBM and the Warehouse Accelerator will allow partners to extend their existing applications built on top of Informix and to introduce new operational analytic capabilities for their clients. In terms of direct sales, however, IBM is targeting smart metering (and therefore utilities and energy companies) with the Informix Warehouse Editions. This application has both operational and analytic requirements. In the former case, for example, you need to recognise outages and handle them in a timely manner, while in the latter case you need analytics for planning purposes (for example, at what times of day do we need to ramp up power production), marketing (what incentives can we provide for more efficient use of resources) and fraud detection, to quote just some examples.

The company targets telecommunications, public sector and retail as sectors in addition to smart metering. It is also examining which markets might be most suitable to use Informix's geospatial capabilities both as a stand-alone feature and in conjunction with time stamps.

While Informix has customers numbered in the tens of thousands and there are many Informix customer references on the IBM web site (the most recent being Cisco) these are predominantly for Informix per se rather than the Warehouse Accelerator and those that we do know of in this category do not want their names publicised.

The Informix Warehouse Accelerator enables dynamic query processing in-memory; together with parallel vector processing and advanced compression techniques; along with a column-based approach to avoid any requirement for indexes, temporary space, summary tables or partitions. In other words it is entirely suitable for supporting analytic applications because the lack of the features mentioned means that administration is both minimised and consistent across transactional and analytic environments.

Typically, the Warehouse Accelerator will be implemented on the same system as the relevant transactional environment. When this is the case you use Smart Analytics Studio, which is a graphical development tool, to define the data (and its schema) that you want to query and the Warehouse Accelerator will automatically offload this data, which is now stored separately from the OLTP environment. It is processed in its own memory space so that there is no conflict with the operational aspects of the environment and transactional performance will not be impacted. Note that there is no need to change your existing business intelligence tool(s).

The optimiser has been specifically designed to support both transactional and analytic workloads when a hybrid environment is being deployed. The optimiser knows what data is in the data mart and what is not: it will determine whether the query can be satisfied by the Accelerator and, if so, it routes the query there. If not, it will choose to execute the query within Informix. Now, if a query saves the result into a temporary table as part of the Select statement, as is often done by certain BI tools, then the Accelerator can speed up that portion of the query.

The smart metering market, at which the Informix Warehouse Editions are mainly aimed, is still in its very early stages. However, whether it is for smart metering or not the ability that Informix offers, to perform serious analytics on the same platform as transactional functions, is a strong one. The way that the system has been designed is not simply as one system for two different functions - which we do not regard as ideal - but, effectively, as two different systems on the same box and with the same administration; both linked by some clever software and purpose-built optimiser functions. Thus from a theoretical point of view, Informix has every chance of success within its target market. The danger is that while IBM has recently ramped up its development and marketing of Informix, this may not last if returns are not seen in the short to medium term. Informix is very much IBM's third database, after DB2 and Netezza, and it is important that momentum is maintained.

In addition to the normal sorts of training and support services you would expect from any vendor, IBM offers business services (application innovation, business analytics, business strategy, functional expertise, midmarket expertise), IT services (application management, business continuity and resiliency, data centres, integrated communications, IT strategy and architecture, security), outsourcing services (business process outsourcing and IT outsourcing and hosting), asset recovery, hardware and software financing, IT lifecycle financing and commercial financing.

InfoSphere Discovery is an enabling tool rather than a solution in its own right so it is horizontally applicable across all sectors. It will be particularly useful where it is necessary to understand business entities (for example, a customer with his orders, delivery addresses, service history and so on) and process those business entities as a whole. Notable environments that require such an approach include application and database-centric data migrations, master data management and archival.

We have no doubt that IBM has many successful users of InfoSphere Discovery. However, you wouldn't know that to judge by its web site, which includes just two case studies where customers are using the product - CSX and FiServ - but in neither paper is the use of InfoSphere Discovery discussed; the product is simply listed as one of the IBM products in use.

In addition to providing conventional data profiling capabilities (finding and monitoring data quality issues) Discovery supports the discovery of orphaned rows, scalar relationships (simple mappings, substrings, concatenations and the like), arithmetic relationships between columns, relationships based on inner and outer joins, and correlations for which cross-reference tables are generated. Cross-source data analysis is available both to discover attribute supersets and subsets, and to identify overlapping and unique attributes. In the latter case there is a visual comparison capability that allows you to compare record values from two different sources on a side-by-side basis. In addition there are automatically generated source rationalisation reports that compare data sources to one another. Further features include support for filtering, aggregations and if-then-else logic, amongst others.

There is also a Unified Schema Builder designed specifically to support new master data management, data warehousing and similar implementations that includes precedence discovery, and empty target modelling and prototyping. There are also facilities for cross-source data preview, automated discovery of matching keys (that is, a cross-source key for joining data across sources), automated discovery of business rules and transformations across two or more data sets with statistical validation, and automated discovery of exceptions to the discovered business rules and transformations.

This is what we wrote in our 2012 Market Report on Data Profiling and Discovery: "since our last report into this market IBM has acquired Exeros, which was market leading for discovery purposes at that time. It should therefore come as no surprise that IBM offers the best understanding of relationships of any product that we have examined. For pure profiling capabilities IBM InfoSphere Discovery is good without being outstanding but it is clearly one of the market leaders when discovery is required alongside profiling." That view remains unchanged: in support of MDM, migration, archival and similar environments Discovery is clearly the leading product in the market.

In addition to the normal sorts of training and support services you would expect from any vendor, IBM offers business services (application innovation, business analytics, business strategy, functional expertise, mid-market expertise), IT services (application management, business continuity and resiliency, data centres, integrated communications, IT strategy and architecture, security), outsourcing services (business process outsourcing and IT outsourcing and hosting), asset recovery, hardware and software financing, IT lifecycle financing and commercial financing.

IBM InfoSphere Optim Test Data Management is horizontally applicable across all sectors and is marketed directly by IBM. It is particularly useful in DevOps environments where development and operations are collaborating with one another and it also especially complements agile development methodologies.

IBM also has partners marketing specific solutions based on IBM InfoSphere Optim Test Data Management. For example, TouchWorld offers a solution for test data for SAP applications.

IBM’s InfoSphere Optim Test Data Management Solution has a global client base with over a thousand clients worldwide across a variety of industries. DevOps and the need for Continuous Testing are currently driving more demand for an effective test data management solution. Data masking for non-production environments is another requirement creating demand. Customers using IBM Optim Test Data Management include Nationwide Insurance, Conway Inc., GEICO, CSX Corporation, Allianz Seguros, Cetelem (part of BNP Paribas), Dignity Health and HM Land Registry.

IBM's InfoSphere Test Data Management solution provides capabilities to: discover and understand test data, discover sensitive data, subset data from production, mask test data, refresh test data and analyse test data results. This integrates with the IBM Rational Test Workbench as well as other leading testing suites such as HP Quality Center. You can also use the IBM Rational Service Virtualisation Product (previously Greenhat) and IBM’s Rational Urbancode Deploy in conjunction with this offering. This service virtualisation is important when (some) test data is derived from external sources or is otherwise not easily available.

Notable features of IBM InfoSphere Optim Test Data Management include a range of data masking options, very advanced discovery capabilities, a data comparison capability, data sub-setting based on an understanding of data relationships (to ensure that test data is representative) and support for multi-sized data subsets. InfoSphere Optim Test Data Management can generate a completely anonymous set of data, but this requires a specification of the original data set, which can be a barrier for some clients. Note that this differs from synthetic generation that is generated from a profile of the data, as opposed to being generated from a base dataset. It is likely that IBM will introduce synthetic generation at some point in the future.

There are actually three phases to deployment of IBM InfoSphere Optim Test Data Management: move, edit and compare. In the move phase you extract, copy and move the data (which may come from multiple sources) not necessarily just for creating test data but also to support data migration and data ageing. In the edit phase you can view, and edit if necessary, the extracted data, which can be in any arbitrarily complex schema. In the compare phase you can compare data from one set of source tables with another either online in the Optim GUI or in reports. Compare enables automated comparison of two sets of data, then enables users to do things like compare data after a test run with the baseline 'before version' or track database changes or compare different data sources. The three phases, deployed together, enable organisations to easily conduct iterative testing, for faster and more comprehensive testing.

The product runs on Windows, Linux and UNIX platforms and supports not just IBM’s own database products (DB2, Informix et al) but also leading third party database products such as Oracle, SQL Server and Teradata.

In addition to the normal sorts of training and support services you would expect from any vendor (which includes extensive online resources), IBM offers business services (application innovation, business analytics, business strategy, functional expertise, mid-market expertise), IT services (application management, business continuity and resiliency, data centres, integrated communications, IT strategy and architecture, security), outsourcing services (business process out-sourcing and IT outsourcing and hosting), asset recovery, hardware and software financing, IT lifecycle financing and commercial financing.

IBM has two strategies with respect to InfoSphere Streams. In the first place it wants to build a community of users, which is why it introduced the Quick Start Edition during 2013. Secondly, it wants to build ecosystems of applications, and partners building those applications. In this case, it is focusing on the telecommunications sector in the first instance, but expects to expand into other vertical markets as time progresses.

While IBM already has a number of partners for InfoSphere Streams, few of these will be known to readers. The most notable exception is with respect to IBM’s partnership with Datawatch. The latter is not a development partner but instead provides integration capabilities to external sources of data such as message queues—of course, IBM supports its own WebSphere MQ—but Datawatch provides the ability to access data from a variety of third party sources.

InfoSphere Streams has a diverse range of users. Early adopters of the technology included hospitals (neo-natal units), wind farms, and oil companies predicting the movement of ice floes, as well as a number of scientific deployments. More recently IBM has identified a number of repeatable and more commercially oriented use cases that it is now focusing on. In the short term, the company is focusing on the retail sector, particularly around data breaches, and the financial sector for fraud prevention and detection as well as risk analytics. Telecommunications is also a focus area but there are many others where InfoSphere Streams might be applicable, such as preventative maintenance and other applications deriving from the Internet of Things.

InfoSphere Streams is both a development and runtime environment for relevant analytics. In the case of the latter the product will run on a single server or across multiple, clustered servers depending on the scale of the environments and ingestion rates for real-time processing.

As far as development is concerned, when the product was originally launched it used a language called SPADE (stream processing application declarative engine) but it now supports SPL (stream processing language), which is SQLesque (indeed, the product supports IBM’s Big SQL). There is a conversion facility from SPADE to SPL. However, for most practical purposes all of this is under the covers as the product includes an Eclipse-based drag-and-drop graphical editor for building queries that business developers, in particular, will generally work with. Using this you drag and drop operators while the software automatically syncs the graphical view you are creating with the underlying (SPL) source code. Debugging capabilities are provided for those that want to work directly with SPL.

As an alternative you can create predictive models using SPSS Modeler and import these into the Streams environment via PMML (predictive modelling mark-up language) or using the native SPSS Modeler models and scoring libraries. The environment also supports both Java and R, the statistical programming language, and text analytics via natural language processing (which is good for sentiment analysis, intent to buy analyses and so forth). Finally, there is support for both geospatial and time-series capabilities with the former supporting location-based services and the latter providing a variety of analytic and other functions (including regressions) that are particularly relevant where data is time-stamped, which is especially relevant to the Internet of Things.

For data input, InfoSphere Streams supports MQTT (Message Queue Telemetry Transport), which is a lightweight messaging protocol that runs on top of TCP/IP, as well as WebSphere MQ and the open source Apache ActiveMQ. Other messaging protocols and feeds are supported through a partnership with Datawatch and there is also a RESTful API. There is also support for accessing data from back-end data sources such as the various IBM PureData products as well as third party data warehouses like HP Vertica.

For presentation purposes the product comes with a number of pre-defined graphical techniques that can be used to visualise information and these can be dynamically added at runtime, as required. In addition, you can use both IBM and third party data virtualisation products such as, in the case of IBM, Watson Explorer. There is also a facility to visually monitor applications while they are running.

In addition to the normal sorts of training and support services you would expect from any vendor (including extensive online resources), IBM offers business services (application innovation, business analytics, business strategy, functional expertise, mid-market expertise), IT services (application management, business continuity and resiliency, data centres, integrated communications, IT strategy and architecture, security), outsourcing services (business process outsourcing and IT outsourcing and hosting), asset recovery, hardware and software financing, IT lifecycle financing and commercial financing.

IBM delivers the PureData System for Analytics through its global direct sales force. As well as plentiful reference customers in Telecommunications, Finance & Banking, Retail and Marketing Analytics, there does not seem to be any industry that is not penetrated to at least some extent.

Users span a wide set of analytics and reporting use cases, with particular growth in in-database analytics since the introduction of a wide range of analytics capabilities; for example support for R and a partnership with Revolution Analytics for their strong R-based algorithms.

Customers who adopt a 'logical data warehouse'—an extended data management architecture, comprising data in multiple formats and originating from multiple sources—sometimes describe the role of the PureData System for Analytics as the “relational data lake” in that architecture. Others are using it in more traditional warehousing roles; as data marts, multi-subject-area warehouses, central warehouses and any combination of these.

The major industries for the PureData System for Analytics appear to be Telecommunications, Finance & Banking and Retail, although there are also significant customer references in Marketing Analytics and most industries appear to be penetrated to at least some extent.

IBM is experiencing strong growth within existing large corporate accounts, as data volumes expand and more use cases are implemented. There is also strong growth in new accounts.

This suggests that the basic proposition—faster complex, analytic queries—has resonance in a wide range of situations. While scalability (see Technology) allows support of very large databases, the low entry point (1/4 rack) means that the PureData System for Analytics solution is also available to SMBs. Our research indicates that the 1/4 rack machine was specifically added to appeal in European markets where data volumes are typically lower and there is a greater tendency to start small and incrementally build.

The PureData System for Analytics is shipped in units of 1/4 rack, 1/2 rack, one rack, two racks, four racks and eight racks, which provides capacity for over 700Tb of user data on a single appliance.

The shared-nothing MPP architecture has been refined over a number of generations by IBM, based on the Netezza multi-core processors and patented FPGA (Field Programmable Gate Arrays) co-processor architecture. The FPGAs provide a unique way to stream query results from storage, avoiding pre-loading into memory. This has long been a successful technology for Netezza and, subsequently, IBM.

Recent iterations of the system have added additional performance-enhancing refinements such as columnar compression, snippet result caching, and large memory on processors. The PureData System for Analytics now ships with over 200 in-database analytic functions, including geo-spatial functions (provided by partner ESRI).

One further valuable addition is the ability for users to licence, and pay for, only part of the capacity of an appliance they have installed, so they have expandability built-in and can match investment more closely to their needs, eliminating the 'chunky' cost of acquisition. This feature is available from one rack and upwards. The physical 1/2 and 1/4 rack options provide downward scalability.

IBM offers a huge range of services, including application development, analytics, strategy, functional expertise, IT services, outsourcing, hosting and financing.

Since most PureData System for Analytics sales are into organisations with existing, transferable SQL database skills, adoption of the new platform is rarely a major issue.

Apart from the focus on particular types of environment, as discussed, IBM does not have any particular vertical focus. Indeed, given the company's size you would expect it to be all-encompassing. However, it does offer specialised data models for a number of specific sectors, notably for banking, financial markets, healthcare, insurance, retail and telecommunications. It also offers a more generic pack for customer insight, market and campaign insight and for supply chain insight.

IBM has been a leading vendor in the (enterprise) data warehouse market since its inception. As such it has thousands of customers, both famous and not so famous. Historically it focused on the high end of the market but today it has offerings that scale down to less than 1TB.

The IBM PureData System for Operational Analytics is available in "T-shirt" sizes: extra small, small, medium, large, extra-large, and so on. At the bottom end the system consists of a "Foundation" module; and then you can add data modules, which each take up 1/3^rd of a rack in any number you like up to 6 racks (that is, 18 data modules) - where each data module provides 62.4TB of raw data capacity. As far as failover is concerned, one failover module is required for each three data modules, with the proviso that you must have a failover module in each rack.

Notable features of the PureData System for Operational Analytics include advanced compression, piggy-back scans and multi-dimensional clustering. With v10.1 of DB2 you also get zigzag joins, continuous ingest and time-travel queries. Once DB2 v10.5 (released April 2013) is available as a part of this system (which we expect in due course),  this will provide BLU Acceleration, which includes columnar storage, dynamic in-memory caching, parallel vector processing, data skipping and even more advanced compression. All of these features are designed to significantly improve performance.

Over the last decade or so there has been a significant shake-up in the data warehousing space. However, a clear pattern has now emerged. There are:

• Merchant database vendors that believe in a one-size-fits-all approach to both transaction processing and warehousing.
• IBM: a merchant database vendor that believes that you cannot get the best performance characteristics from a system that is supposed to cater to both transaction processing and analytics and which has therefore created specialised bundles for specific purposes, such as the PureData System for Operational Analytics.
• Traditional specialist data warehousing vendors that will compete with the PureData System for Operational Analytics.
• A host of newer (and some not so new) vendors that are really offering data marts rather than something that is suitable for use as an enterprise data warehouse.

In other words, the IBM PureData Systems for Operational Analytics faces exactly the same competitors as IBM data warehousing has faced historically. IBM has been successful over the last twenty years in acquiring a significant slice of the data warehousing market and we so no reason why that should change now.

In addition to the normal sorts of training and support services you would expect from any vendor, IBM offers business services (application innovation, business analytics, business strategy, functional expertise, midmarket expertise), IT services (application management, business continuity and resiliency, data centres, integrated communications, IT strategy and architecture, security), outsourcing services (business process outsourcing and IT outsourcing and hosting), asset recovery, hardware and software financing, IT lifecycle financing and commercial financing.

Although IBM states that its InfoSphere Guardium products can help organisations streamline operations regardless of their size, these products are generally used by large enterprises, often multinationals with worldwide operations. It claims this product family scales to support tens of thousands of databases in an organisation, including those on-premise, in the cloud or in Hadoop environments for big data.

The Guardium family of products can be purchased in a modular fashion, allowing organisations to choose individual products from within the portfolio, as well as to mix them with components from any other vendor.

These products are aimed at large enterprises that face pressing security and regulatory compliance concerns. In a video on its website, IBM states that its InfoSphere Guardium products are used by some 600 organisations worldwide, including the majority of the top banks, managed healthcare providers, retailers, insurers and telecommunications companies globally.

Designed to protect trusted information in databases, InfoSphere Guardium supports all major database management platforms and protocols on all the major operating systems, as well as a growing range of file and document sharing environments. It also supports a number of enterprise business suites, including those from Oracle and SAP.

InfoSphere Guardium is based on a multi-tier architecture, using connector appliances to gather data from systems under management, which are then collected in a central management console where the data is aggregated and normalised, providing an audit trail and secure management and enforcement of policies. No changes are required to the configurations of systems under management and all functions are managed in one central repository. For creating greater security intelligence, Guardium interfaces with a number of other systems, including LDAP directories for access control, email, change ticketing and SIEM systems.

In addition to the normal sorts of training and support services you would expect from any vendor, IBM offers business services (application innovation, business analytics, business strategy, functional expertise, mid-market expertise), IT services (application management, business continuity and resiliency, data centres, integrated communications, IT strategy and architecture, security), outsourcing services (business process outsourcing and IT outsourcing and hosting), asset recovery, hardware and software financing, IT lifecycle financing and commercial financing.

IBM delivers InfoSphere Information Server through its global direct sales force.

As a product that is around 20 years old (InfoSphere DataStage, as it was then, was originally developed by Ardent, which was acquired by Ascential, which in turn was acquired by IBM) there are thousands of users of InfoSphere Information Server and there are a number of case studies published by IBM on its web site, including the company’s own implementation in the office of the CIO, as well as banking customers, universities and others.

Perhaps the most significant aspect of InfoSphere Information Server, outside its role of physically moving and transforming data at scale (including big data integration and grid deployments), is the extent to which it plays a part in information governance. Thus the product includes the ability to define governance policies and rules, a data governance dashboard, integration with InfoSphere Master Data Management, Blueprint Director (used to model and view the integration and governance architecture) and the Information Governance Catalog (previously Business Information Exchange), which provides a business glossary and metadata workbench. There is also a new lineage viewer.

Alongside core capabilities (for example, new connectors to the cloud) the emphasis in the data integration market at present is very much on self-service capabilities, just as the same is true for business intelligence. InfoSphere Information Server provides this with InfoSphere Data Click, which is a simple web-based interface for on-demand integration, both between conventional sources but also for access data lakes (integration with Hadoop is built into the product). However, self-service is not only about the user experience, it is also about automating functions that underpin what the user can do. Thus, for example, Information Server supports the idea that you can move data into, say, an IBM data warehouse using ETL at one time of the day and ELT at another time of the day. However, this has to be manually scheduled and it would be preferable if there was an optimiser built into the product to automate this and other processes. We understand that such an optimiser is planned. We should also add that this is not a criticism—other vendors in this space do not, typically, have even the existing level of automation that IBM provides. Other self-service capabilities include smart hover (which provides contextual details about an asset when you hover your mouse over it) and semantic search capabilities. There are also new facilities in the latest version (11.3) to support cross-team collaboration.

In addition to the normal sorts of training and support services you would expect from any vendor (which includes extensive online resources), IBM offers business services (application innovation, business analytics, business strategy, functional expertise, mid-market expertise), IT services (application management, business continuity and resiliency, data centres, integrated communications, IT strategy and architecture, security), outsourcing services (business process out-sourcing and IT outsourcing and hosting), asset recovery, hardware and software financing, IT lifecycle financing and commercial financing.

Rational Requirements Composer is available, and supported, globally. Licenses can be bought through normal IBM channels. However it can also be downloaded from jazz.net and potential users can try it out in a web-based 'sandbox' environment, from a web browser, Eclipse client, Visual Studio client etc.; and potential collaborators can be invited to join in the trial. 

Requirements Composer is aimed at Agile developers in the enterprise, especially those aiming for 'agile at scale'. However, it is available to a much wider community than this via jazz.net on the web.

Rational Requirements Composer is a web application running on a wide range of desktops: Windows or various flavours of Linux. Its server runs on various flavours of Linux, on Windows Server, Sun Solaris, AIX IBM Power System and VMWare. It uses a wide range of databases to hold requirements data and metadata; Apache Derby is included but D2, Oracle and SQL Server are also supported. Apache Tomcat application server is included, but WebSphere is also supported

It supports the Open Services for Life-cycle Collaboration (OSLC) specifications, so it can integrate with a range of development tools from IBM (and, potentially, other vendors).

What this all means is that Rational Requirements Composer isn't dependent on being deployed in all-IBM environments, which might not have been so true of past IBM tools.

IBM offers all the services (consultancy, training etc.), available across the globe, that one might expect to need in support of this product.

Paricularly noteworthy is the Jazz Community at jazz.net, which represents IBM's next generation of interactive, collaborative tools for delivering buisiness outcomer from smarter automated systems. jazz.net is a lot more than a conventional product support site  and according to IBM "we use our products to build our products, right here at Jazz.net. You can track our plans, talk to the developers, and try our latest stuff. Help us build the tools that you're dreaming about!". There does, indeed, seem to be a buzz around jazz.net.

Rational Requirements Composer is also part of the OSLC (Open Services for Lifecycle Collaboration) community for tool integration

A wide range of training courses, both classroom and e-Learning, for Rational Requirements Composer are available from IBM and its partners.