JSON Archives

YAML Editing Tools

May 2, 2024/in Development, JSON, Tools, XML /by Erin Cavanaugh

YAML is increasing in popularity due to its combination of human readability, simplicity, and versatility. Often used in configuration files and for data serialization, YAML can be used alongside JSON and XML in modern systems. It makes sense, then, for developers to choose an IDE that supports all three standards.

New YAML tools in XMLSpy add to comprehensive support for XML and JSON development, giving users the flexibility to choose the most suitable serialization format for their specific use cases and preferences.

Let’s take a look at YAML support in the XML and JSON editor.

decorative image: software developer working on code

Tags: JSON, XML, XMLSpy, YAML

AI Tools for XML and JSON Development

November 13, 2023/in AI, JSON, XML /by Erin Cavanaugh

The explosion of AI tools has made a significant impact on the field of software development – not by replacing software engineers as some have predicted – but by actually increasing their value by freeing them to focus on higher-level tasks. By automating low-level code generation, for instance, AI increases development speed and opens the doors to deeper innovation.

To give developers the AI tools they need to realize these productivity and creativity gains, Altova has integrated AI functionality in XMLSpy for XML and JSON editing tasks.

Here’s how the XMLSpy AI Assistant works.

AI Integration & PDF Data Mapping in Version 2024

October 24, 2023/in AI, Data Integration, JSON, XML /by Erin Cavanaugh

Version 2024 of Altova Software introduces brand new AI Assistants in multiple products as well as long-awaited support for PDF data integration in MapForce. Other features include Markdown editing support, split output preview for business report creation, support for new XBRL standards, and much more.

Let’s take a look at the highlights.

Decorative image with imagery symbolizing AI to announce new product features

5 Reasons to Choose a Graphical JSON Schema Editor

September 25, 2023/in Development, JSON /by Erin Cavanaugh

The advantages of JSON as a lightweight, human-readable, interoperable data format have led to its widespread adoption in various domains, including web development, mobile app development, and backend services. Many programming libraries and frameworks provide built-in support for JSON parsing and serialization.

That said, most applications still benefit from or require validation of client-submitted data. Enter the JSON Schema spec, which lets you describe the structure of JSON data for a particular application, for both documentation and validation purposes.

Though JSON Schema code is by design human-readable, building a complex schema with nested and repeating sections in a text-only editor becomes time consuming and error-prone quickly. Let’s look at five ways a graphical editor is a must-have for JSON Schema development.

Tags: JSON, JSON Schema, XML JSON, XMLSpy

New Tools for Large XML and JSON Documents

May 24, 2023/in Development, JSON, XML /by Erin Cavanaugh

Code editors with a graphical option in addition to the traditional text view are great for developers working with XML or JSON. Graphical editors like Grid View in XMLSpy provide a visual representation of the JSON or XML structure, making it easier to understand, navigate, and edit complex data hierarchies.

XMLSpy introduced Grid View years ago and has been improving on it ever since. The most recent update includes the ability to split the editing pane horizontally or vertically, which is especially handy when working with large documents.

Let’s see how it works.

Decorative image of a computer monitor in a dark-lit office setting

Tags: Grid view, new features, XMLSpy

Split Mode in Grid View and More in v2023r2

April 12, 2023/in Data Integration, Development, JSON, XML /by Erin Cavanaugh

The latest release of Altova software includes ongoing enhancements for graphical XML and JSON editing in XMLSpy Grid View, new tools for working with XPath/XQuery, support for integrating VDA EDI data in MapForce, and much more.

Here’s a quick overview of all the new functionality being introduced.

Decorative image of a female viewing computer programming on a large screen

Tags: MapForce, new features, video, What's new, XMLSpy

Analyze JSON Data with Filters, Formulas, and Charts

May 4, 2022/in JSON, Tools /by David McGahey

Software developers and other data professionals often need to examine new data instances before designing processes for efficient production. As JSON becomes a more popular format for data exchange, the tradeoff for smaller data payloads can be loss of clarity of the underlying data structure.

XMLSpy has supported viewing, modeling, and editing JSON files since 2010 and includes rich tools to analyze JSON data, including applying filters, formulas, and charts.

Let’s take a look.

Tags: data analysis, Grid view, JSON, tools, XMLSpy

Transforming and Converting Protobuf

February 10, 2022/in Data Integration, Database, JSON, protocol buffers, XML /by David McGahey

MapForce supports mapping protocol buffers (Protobuf) to and from other structured data formats as mapping sources or targets. In the constant quest for more efficient ways to transfer, manipulate, and manage large structured data sets, Google has created a language- and platform-neutral data format similar to XML, but smaller, faster, and simpler than even JSON data. Tools are available to generate and work with Protobuf using Java, Python, C++, C#, Ruby, and other programming languages.

The structure of any Protobuf message is defined in a .proto file that defines each field name and value type. Altova MapForce lets users drop these .proto files into a data mapping as a source or target along with any other data, including XML, JSON, relational databases, Excel, flat files, REST and SOAP web services, and others. .proto files versions 2 and 3 are supported.

A MapForce data mapping creates compatibility between existing XML, JSON, database or legacy data formats and new applications leveraging the efficiency of Protobuf.

Data Mapping NoSQL Databases

November 26, 2021/in Database, Database mapping, JSON /by David McGahey

NoSQL databases are non-tabular databases that store data differently than traditional databases made up of relational tables. Two of the most popular NoSQL databases, MongoDB and Apache CouchDB, store data as collections of BSON (binary JSON) and JSON documents. These databases leverage flexible JSON schemas and scale easily with large amounts of data and high user loads.

Altova MapForce has long supported data mapping all popular relational databases and now also includes native support for data mapping NoSQL databases. MapForce includes functionality for inserting, extracting, filtering, and ordering NoSQL data. Let’s look at an example.

New Features in XML and JSON Grid Editors

November 9, 2021/in Development, JSON, XML /by Erin Cavanaugh

XMLSpy is known for its graphical editing views, which make working with and editing XML and JSON more straightforward – and therefore increase developer productivity.

Both XML Grid and JSON Grid have received major updates in recent releases, and we’re committed to making these graphical views as feature-rich and effective as possible through continuous improvement. As such, we rely on and appreciate feature requests and developer feedback. The latest release of XMLSpy provides some great updates in response to this input.

Tools for JSON Comments and JSON Lines

September 1, 2021/in JSON, Tools /by Erin Cavanaugh

Altova XMLSpy and MapForce JSON tools have long supported JSON and JSON5 for editing as well as data mapping and conversion. As new JSON formats arise in response to real-world usage, the support in these tools is expanding.

This article will help explain the advantages of two newer formats – JSON Comments and JSON Lines – and show how to use them in XMLSpy and MapForce.

How to Learn a New Programming Language this Summer

July 21, 2021/1 Comment/in Development, JSON, Software, Technology, XQuery /by Alexander Falk

What better goal to set for your summer than learning a new programming language? Forget the “beach books” this summer and set your sights on diving into a new coding language – but which one should you pick?

As reported by EDC there are approximately 24 million professional software developers on Earth, and almost 700 notable programming languages (according to Wikipedia). I would wager a bet there is a staggeringly equal number of places online where you can learn one programming language or another. Whether you are already one of those 11 million coding experts or a newbie to programming, there is a plethora of information out there to sort though.

For the purposes for this blog post, we will certainly look at the world’s most-widely-used programming languages in 2021, but also at important languages for data manipulation and querying, so we’ll discuss: C and its derivatives (C++, C#, and Objective-C), Java, Python, R, JavaScript, Ruby, SQL, and XQuery.

Deciding where to start depends entirely on the kind of development scenarios you have in mind, so we’ve broken things down for you to make it easier. It doesn’t matter if you are a seasoned programmer looking to add a new language to your repertoire or a novice who doesn’t know the difference between C, C++, Objective-C, or C# yet. We have assembled a list of explanations to help you choose which language you may want to conquer next.

Updated Support for the Latest JSON Schema Versions

June 1, 2021/in JSON, Servers, Software /by Erin Cavanaugh

The latest release of Altova desktop developer tools and server software products introduces support for the newest JSON Schema versions in all JSON-enabled products. This adds to support for previous versions of the JSON Schema specification.

An Easy Way to Test HTTP Requests During Development

January 20, 2021/in Development, JSON, XML /by Erin Cavanaugh

Web and web services developers often need to send HTTP requests – whether for testing APIs, testing REST and SOAP web services, or managing web sites.

XMLSpy makes it easy to send and receive HTTP requests directly in the XML and JSON editor during development with its HTTP Window and WADL/WSDL Import Wizard, a great time-saving tool for debugging web services.

New JSON Editing Features

November 18, 2020/in Development, JSON, Tools /by Erin Cavanaugh

In Version 2020, Altova introduced an entirely new approach to JSON editing in XMLSpy called JSON Grid View. This visual JSON editor offers a graphical representation of the JSON document structure that is immediately easier to understand and work with than the corresponding JSON code in a text editor, especially for long, complex documents with multiple nested levels of arrays and objects.

With each release we improve upon the list of unique editing tools in JSON Grid. Let’s take a look at some important updates announced in Version 2021, which was released in October of 2020.

Tags: JSON, JSON edito, new features, XMLSpy

New tools for JSON, EDI, SWIFT, and XBRL in Version 2021

October 14, 2020/in Development, EDI, JSON, XBRL /by Erin Cavanaugh

As standards-based technologies and their applications evolve, so does the functionality that users require from developer tools. In our latest release, we’ve added new ways to work with XBRL, JSON, EDI, and more that help Altova customers work faster and more efficiently.

Let’s take a look at some highlights from the Altova Software Version 2021 release.

Tags: EDI, MapForce, new features, StyleVision, SWIFT, v2021, XBRL, XMLSpy

Examine, Edit, and Validate .x3d Files

August 21, 2020/in Applications, JSON, XML, XSL /by David McGahey

The .x3d file format is an ISO/IEC xml-based standard for representing 3d images. Also known as X3D, .x3d files originated as a way to represent 3D images for computer graphics. Over time applications grew from virtual reality (VRML), to CAD design, architectural rendering, 3D printing, medical and anatomical imaging, representation of molecular chemical structures, human animation, and more.

The Web3D Consortium maintains the X3D standard, with XML Schemas documented here. Current versions include definitions of more than 250 XML elements, over 100 simple types, 70 complex types, and more than 40 defined node groups.

It takes that much complexity to accurately describe anything from a precisely manufactured machine part to a richly textured virtual environment, especially considering lighting, texture, and point of view! XMLSpy, the world’s most popular tool for modeling, editing, transforming, and debugging XML and JSON related technologies, supports .x3d files. XMLSpy offers standards-based tools for anyone who may need to examine, exchange, consume, or validate .x3d files. Let’s look at some examples.

Get to Know Your Favorite XML Editor Again [Video]

May 20, 2020/in Applications, JSON, XML /by Erin Cavanaugh

For more than a decade XMLSpy has been a leading choice for all things XML. But over the years we’ve broadened the scope by adding hundreds of new features, supported standards, and innovative tools to make development faster and easier.

This new overview video offers a quick, high-level look at the various tools and options available, including functionality for JSON, XPath/XQuery, HTTP testing, debugging, back-mapping, and more.

Check out the XMLSpy demos page for more videos and in-depth tutorials on the JSON Grid Editor, XPath and XSLT for JSON, comparing XML files, and more.

Get access to all these features by upgrading to the latest version of XMLSpy or downloading a free, 30-day trial.

Tags: demo videos, JSON editor, XML Editor, XMLSpy

Web Service Data Integration

May 11, 2020/in Data Integration, JSON, Tools /by David McGahey

In a previous post we wrote that every data integration and reporting task needs to start with a clear understanding of the source data. Using grid view in XMLSpy, the industry-leading XML and JSON editor, we analyzed JSON data for 5-day weather forecasts retrieved from a Web service.

Continuing with our earlier scenario, we’ll use MapForce, the award-winning, graphical data mapping tool for any-to-any conversion and integration, to map the forecasts for a series of major cargo shipping ports into nicely formatted Excel documents. We’ll want to highlight any predicted high winds or heavy rainfall that could cause delays by interfering with cranes loading and unloading containers, or slowing ships entering and exiting the harbors.

New Tools for XML and JSON Editing

March 13, 2020/in Development, JSON, XML /by Erin Cavanaugh

The latest release of XMLSpy adds several user-requested features for working with JSON and XML, as well as new functionality for debugging XPath, updated standards and database support, and more.

Tags: JSON, JSON editor, new features, XML, XML Editor, XMLSpy

Data Mapping JSON Lines

February 24, 2020/in Data Integration, Database mapping, JSON /by David McGahey

The JSON data format continues to evolve as an open standard as it is creatively applied to new data interchange requirements. JSON Lines, defined at http://jsonlines.org/, is a convenient text format for storing structured data where each record is a single line and a valid JSON object. JSON Lines handles tabular data and clearly identifies data types without ambiguity. This allows records to be processed one at a time, which makes the format very useful for exporting and sending data.

Altova MapForce supports data mapping JSON Lines as either a data source or target. Let’s look at a mapping project to extract records from a database table and map to a JSON Lines file for output.

Embedding Images in XML

January 15, 2020/in Development, JSON, XML /by Erin Cavanaugh

One of the really cool features added to XMLSpy a few years ago based on customer requests is the ability to embed external files – such as images – directly in an XML document as encoded text. This gives you the option to package all required data from various external files together in one large XML document. The functionality is also available for embedding images in JSON documents.

Let’s take a look at how easy it is to accomplish this in the XML and JSON editor in just a few steps.

Photo for embedding images

Tags: JSON, JSON editor, XML, XML Editor, XMLSpy

Version 2020 Revolutionizes JSON Editing

October 9, 2019/in JSON, XBRL, XML /by Erin Cavanaugh

We’re introducing several exciting new tools for JSON development in Altova Software Version 2020, but there’s much more too. Support for the XULE XBRL standard, comparing CSV against database content, and updated database support are just a few of the new features introduced across the product line.

Let’s take a look at the highlights.

Tags: compare data, JSON, JSON editor, v2020, XULE

Handle HTTP Errors During Automated Data Integration

July 29, 2019/in Data Integration, JSON, Tools /by David McGahey

Data analysts and other professionals often need to generate real-time data through automated execution of data mappings that request Web services and save the results. During automated execution it’s important to gracefully handle any unexpected HTTP error rather than terminate the integration task.

In an earlier post we discussed conditional processing of a REST Web service response to handle HTTP errors, where separate output files were generated for a normal response and an error. Now let’s look at a revised mapping solution for the airport status example to generate a single mapping result file that contains either the requested airport status or a description of the error.

Integrating APIs and Mobile Apps

August 21, 2018/in JSON, mobile development, MobileTogether, XML /by David McGahey

Busy mobile users on the go prefer apps that are convenient and efficient. MobileTogether provides developers with features to seamlessly integrate APIs and mobile apps to combine mobile device functionality with up-to-date information from external sources. This empowers developers to create custom cross-platform native apps that provide a rich and entertaining end-user experience.

Public APIs are a great source of external data to enhance almost any custom mobile app. Developers can combine information from multiple APIs to provide users with better information, faster, in an elegant, integrated package.

APIs are available for almost any kind of information your mobile app may need, from flight tracking to commodity or stock prices to tropical storm tracking.

In this post we’ll look at a GPS app that starts with mobile device geolocation functionality to answer the basic question, “Where am I?” then interfaces with APIs from Google and MapQuest to add a wealth of additional information. We’ll integrate a spatially-aware search engine to locate nearby points of interest as near as a quarter mile radius, all the way to pin-pointing the user’s location in a satellite photograph with a wide-angle view of an entire continent or more.

The Only JSON Dev Tool You’ll Need

May 29, 2018/in Development, JSON, Tools /by Erin Cavanaugh

While XMLSpy might not be the first tool developers think of when they’ve got a JSON development task, XMLSpy includes comprehensive support for working with JSON, JSON Schema, and related technologies.

Over the past few product releases, we’ve added intelligent functionality for editing and converting JSON and JSON5 data to the product. We’ve completed the circle with one-click conversion between XML Schemas and JSON Schemas, as well as sample instance generation and JSON Schema documentation generation. And, most recently, we’ve added support for processing JSON with XSLT, XPath, and XQuery.

Let’s walk through some common examples demonstrating this functionality – and see how these time-saving tools make XMLSpy the only JSON development tool you’ll need.

Developer using JSON tool

Tags: JSON, JSON Schema, MapForce, RaptorXML, XMLSpy

Support for JSON5 in Altova MissionKit, Server Products, and MobileTogether

June 28, 2017/in JSON, Tools /by David McGahey

Altova products have supported JSON for several years. Now, Version 2017 Release 3 of MissionKit and Server products, and MobileTogether Version 3.2 all include support for JSON5 across the product line.

The JSON data format was originally designed to be machine-written and consumed, promoting efficient communication between servers. Usage has expanded and JSON5 is a proposed extension intended to make JSON code easier for humans to write and read. JSON5 extends JSON by adding some ECMAScript 5 features and, like JSON, is a strict subset of JavaScript. Specifically, JSON5 permits inline and block comments, allows long strings to be split over several lines, and defines alternate legal syntax options for quotes and commas. These features are not permitted in standard JSON, so files containing the proposed enhancements are typically identified with the .json5 filename suffix.

This post details specific support for JSON5 in each Altova product.

Learn about JSON5 support in Altova tools

New XSLT Back-mapping Headlines Altova Release

April 5, 2017/in Development, JSON, XML, XSL /by Erin Cavanaugh

It’s time for the latest release of Altova desktop developer tools and server software products, and this one introduces numerous innovative features across the product line, including a brand-new version of MapForce Server called MapForce Server Accelerator Edition for even faster processing of data integration jobs.

Let’s take a look at the highlights of Version 2017 Release 3.

A Typical MapForce Server Use Case

March 2, 2017/in Data Integration, JSON, Tools /by David McGahey

Envision a manufacturing company that controls costs by exploiting a just-in-time assembly process with a very low supply of parts inventory on hand. New customer orders are logged in a sales database and at the end of every day the components needed to assemble that day’s sales are tabulated.

The IT department runs a SQL query to identify the required parts and transforms the list into a purchase order in JSON format to be transmitted to the supply chain.

Sound familiar? Our recent blog series on JSON tools and JSON data mapping were based on this real-life scenario. In this post we describe a MapForce Server use case that automates the repetitive task of generating each day’s purchase order.

JSON Data Mapping and Transformation with MapForce

February 15, 2017/in Data Integration, JSON, Tools /by David McGahey

JSON is a popular format for transferring data between systems thanks to its simple markup, small footprint, and heritage based on the JavaScript programming language. MapForce supports JSON as both an input and output format for JSON data mapping and transformation. For instance, MapForce can extract information from any popular database and produce a JSON file ready for transfer.
The Requirement: Here is an example of a typical need for JSON data mapping: A manufacturing company controls costs by exploiting a just-in-time assembly process with very little parts inventory on hand. New customer orders are logged in a sales database, and at the end of every day the components needed to assemble that day’s sales are tabulated via a query into the database. The required parts will be ordered from suppliers via a purchase order transferred in JSON format.

Tags: data mapping, JSON, MapForce, XMLSpy

Data Mapping REST Web Services

February 24, 2016/in Data Integration, JSON, Tools /by David McGahey

MapForce 2016 Release 2 includes expanded functionality for Web Services data mapping, providing robust support for REST Web services. MapForce accepts XML or JSON as the Web service response, allows definition of parameters, and supports custom HTTP headers. Users may define the Web service interface manually or by importing settings from a WADL file or a URL. Manual definition of REST Web Service Settings lets developers create settings based on a template URL. This is a convenient step when developers test and refine REST calls in a Web browser window, since the URL can be copied from the browser to become the template.

REST Web Services can be a pipeline of information for a data mapping project

New JSON Schema Editor and Data Mapping Debugger Debut in Altova Version 2016

September 25, 2015/in Data Integration, Development, JSON /by Erin Cavanaugh

We are excited to announce details of the latest release of Altova MissionKit desktop developer tools and server software products. Version 2016 includes full Windows 10 compatibility and updated relational database support across the product line, and it also introduces some new features that you simply will not find anywhere else.

XMLSpy 2016 includes the first full featured, enterprise-grade graphical JSON Schema editor. MapForce, our data integration tool, now includes a data mapping debugger that will revolutionize the way you define and test data mapping projects. Let’s take a closer look at these new features.

Altova Version 2016

Learn XPath 3.1 with Altova Online Training

July 1, 2015/in JSON, XPath /by Erin Cavanaugh

We’ve recently updated our free, online XPath Training course to include information on what’s new in XPath 3.1, which is the latest version of the XML Path Language.

XPath 3.1 adds vital new functionality, including support for arrays and maps, functions for processing JSON data, and a collection of new operators and functions.

XPath Training

The XPath Training course covers these in detail, explaining each new function and operator and illustrating its use with helpful examples. The course also provides sample XML files that you can download for hands-on practice as you progress through the chapters.

Access the free XPath 3.0 and 3.1 Training now!

Or check out our other free online course offerings, including XMLSpy Training, XML Schema 1.1 Training, and more.

Tags: Altova Online Training, XPath 3.1

New Support for XPath/XQuery 3.1, WS-Security, and More Debut in R3

February 25, 2015/in JSON, Software, XBRL, XPath, XQuery /by Erin Cavanaugh

Release 3 of the Version 2015 Altova MissionKit desktop developer tools and Altova server software products is now available with updated standards support, customer requested features, and innovative new functionality that will save you time on everything from XPath and XQuery development, to advanced data mapping, to XBRL report rendering and beyond. You can skip right to the good stuff and upgrade now, or click Read More below for all the details.

Now Available: MobileTogether 1.4 with Support for Geolocation, Android 5, iOS 8, and Much More

February 12, 2015/in JSON, Mobile, XQuery /by David McGahey

We are pleased announce version 1.4 of MobileTogether, Altova’s multi-platform mobile development framework that empowers organizations of any size to quickly design and deploy custom enterprise mobile solutions that are accessible on any mobile device.

New features in Altova MobileTogether 1.4

Altova Software Version 2015 Now Available

September 17, 2014/in JSON, Software /by Erin Cavanaugh

We are pleased to announce availability of Version 2015 of Altova’s MissionKit desktop developer tools and server software products! In this latest version, you’ll find updated standards support across the product line, powerful new support for XQuery Update Facility and previewing the layout of XBRL Tables in real-time in XMLSpy, data mapping support for JSON and XBRL Table Linkbases in MapForce, support for HTML fragments in StyleVision, and much more.

Read on for more details below, and visit the What’s New page for more information.

New XBRL Tools for Compliance and Beyond

April 8, 2014/in JSON, XBRL /by Erin Cavanaugh

It can be hard to keep up with all the exciting developments in the XBRL space. Standards are constantly evolving to make compliance easier and deepen the benefits provided by XBRL for internal analysis and reporting. As a leader in XBRL development and validation, Altova is committed to supporting the latest standards, coupling strict conformance with high-performance, easy to use tools. With the latest product launch, Version 2014 Release 2, we’ve added some important enhancements, including significant increases in validation speeds, XBRL Table Linkbase support, and extended XBRL Formula tools. Below is a portion of an XBRL Table generated in XMLSpy, using XBRL Table Linkbase to render revenue calculations in new and productive ways:

Let’s take a look at the benefits of these new features for your XBRL implementation.

Editing, Converting, and Generating JSON

March 12, 2013/in JSON /by Erin Cavanaugh

As the use of JSON as a data transport protocol increases, I thought it would be useful to take a look at JSON support in XMLSpy. There’s been much debate about advantages of JSON vs. XML, but when you boil it down, there are simply some cases for which JSON is the best choice, and others where XML makes more sense. This article on the XML Aficionado blog unpacks this topic quite well.
While you might need to choose between JSON and XML depending on the development task at hand, you don’t have to choose between code editors – XMLSpy supports both technologies and will even convert between the two. Let’s take a look at how that works.
Editing JSON
To make JSON editing as easy as possible, Altova extended its intelligent XML editing features to the JSON editor. If you choose to edit JSON in text view, XMLSpy helps you along with syntax coloring, bracket matching, source folding, entry helper windows and menus, and so on.

Here’s that same JSON file in grid view, which offers a graphical representation of the JSON structure with drag and drop editing. Both views provide JSON syntax checking and advanced error checking features.

Another common requirement is converting XML to/from JSON, which is a one-click option on the XMLSpy convert menu. The JSON data above has been converted to valid XML:

Having JSON editing and conversion functionality directly inside the XML editor is quite useful, especially if you’re using the free Eclipse or Visual Studio integration package.
Check it out and let us know what you think.

Tags: JSON, XML JSON, XMLSpy

Digging deeper with the Twitter API: iPhone 4S vs. Galaxy Nexus

November 8, 2011/in Applications, JSON /by Erin Cavanaugh

We found some interesting data when we dug below the surface of the iPhone 4S vs. Galaxy Nexus debate using the Twitter Search API.In today’s world there is a vast quantity of data available online that can be used for research, market analysis, and competitive intelligence. While “Big Data” can be a problem for those who produce it, store it, and compile it, it is highly beneficial for those of us who are looking for answers.Some of that data is fortunately available to be queried online, and, in particular, there is a vast quantity of data on social media interactions out there.In this article we will explore how to use the Twitter Search API from MapForce, Altova’s data mapping/conversion/integration tool, to aggregate data on recent user submissions (“tweets”) on two highly popular topics – the Apple “iPhone 4S” vs. the “Galaxy Nexus” as the latest hot Android phone – and extract some statistical data about the users engaged in those discussions. One of the benefits of this abundance of data available to us today is that we can query it in interesting ways and extract new meaning from it. While there are undoubtedly many existing services that already provide trends over Twitter topics (e.g., Trendistic), those services only offer very simple trends and do not allow us to query any deeper.But all of the underlying data is available for grabs if you are just willing to learn a tiny bit about web service APIs and how to use them to extract XML data for further processing. As a starting point, let’s use the Twitter Search API to query the stream of recent tweets for the last 100 postings that are about the “Galaxy Nexus”. The Usage Guidelines for Twitter Search tell us that using both words in a query will result in the use of the default operator, which is AND, so we are going to search for posts that contain “Galaxy AND Nexus”. So let’s try that and request the most recent 100 items:

http://search.twitter.com/search.atom?q=galaxy+nexus&rpp=100

If you follow this link, you will get a second window with a lot of raw XML data that is formatted according to the Atom Syndication Format specifications. Alternatively, you could request the data in JSON format, if you wanted to directly process it via JavaScript code by hand, but we will use the XML-based Atom format so that we can easily analyze the data and extract the information we want.Viewing the above search result in a browser is not very user-friendly, so we can take a quick peek at the XML data in our favorite XML Editor using the Open from URL function:As you can see, the data for each entry includes a language code, so for this example we will extract data from this Twitter feed as well as from a second search result on the “iPhone 4S” and combine them into one intermediate XML file for further analysis.Extracting XML data is really easy in MapForce: using the “Insert XML File” option to drop in an XML source, we can again specify the same URL as before. If needed, MapForce will automatically create an XML Schema for the supplied data so we can visualize it and extract information from it:In our mapping we have dropped in two sources on the left side – one using a query string to search for “Galaxy Nexus” and the other to search for “iPhone 4S” – and on the right side we have dropped in a simple XML Schema that will allow us to aggregate our data and analyze it more conveniently going forward. In this case the mapping between the two sides is straight-forward as we are only extracting basic information about the user, the date, and the language of the tweet, but in other applications the mapping could be more complicated and include functions as well as queries to other data sources, databases, or web services…Previewing the resulting XML data can be done directly inside MapForce using the output tab, and this is what we see as a result of our data transformation:Now we can easily use the reporting capabilities of StyleVision to group this data by language within each topic and count the number of posts in each language. We can then report this data in the form of pie charts, which produces the following interesting results:Obviously, this data is highly dependent on the date of execution and time of day, as well as the particular announcements happening about these products, so the numbers will fluctuate quite a bit, but it can be used as a nice monitoring for seeing different language-specific trends. And once this has been set up, the report can be refreshed easily with the click of a button to get a snapshot at that point in time. For more long-term analysis it would of course be necessary to modify the mapping a bit to query more than 100 recent tweets.In this article we have used Twitter’s Search API as one example data source and only looked at language as one unique data point, but there are many more interesting sources of data available online today, and this approach can be used on all of them in a similar fashion.If you want to experiment with other data sources and other kinds of information that you want to extract, we invite you to try for yourself. A free 30-day evaluation version of MapForce is available, and there are no limits on how you can use the other features of Altova’s data mapping and conversion tool for data processing tasks that go beyond analyzing social media trends…

Tags: API, data integration, MapForce, Twitter, XMLSpy

Processing the Groupon API with Altova MapForce

August 2, 2011/2 Comments/in Data Integration, JSON /by David McGahey

We often think of a data integration project as a translation from one singular data input file to some other data set, but Altova MapForce lets you greatly expand the concept of an input file. For instance, the MergeMultipleFiles.mfd example installed with MapForce illustrates how you can use a filename with wildcard characters to merge multiple input files into a single output. A MapForce mapping input doesn’t even need to be a physical file – it can be a URL that returns predictable structured data, like the APIs for popular Web sites like Groupon and many others.

In this blog post we’ll describe how to use Altova tools to retrieve, filter, analyze, and present data available from a Web-based API, using Groupon as an example. If you want to follow along yourself, you will first need to visit http://www.groupon.com/pages/api to request your personal Groupon API client key. The Problem: All Deals Are Local The Groupon Web site and email subscriptions are great for finding deals in your local neighborhood, but what if you’re looking for a deal to use on an upcoming vacation, or for a gift for friends or family across the country? Sure, you could enter each location manually at the Groupon Web page, but that’s so last century. Let’s use the Altova MissionKit to automate things. The Groupon API offers two URL queries that return data in .json or .xml formats: the first returns a list of all Groupon localities (called divisions), and the second returns current deals information for one named division. If we want to see all the deals for more than one division, we need to resolve multiple URLs and aggregate the data into a single result. Yes, MapForce can do that! First We Need a Schema The Groupon API documentation describes the elements that will be returned by our requests, but doesn’t provide an XML Schema. That’s okay, we can use MapForce to generate one. All we have to do is open a new mapping design and choose Insert XML Schema/File, then click the Switch to URL button. Now we can enter the URL to retrieve the Groupon divisions list: When we click the Open button MapForce offers to generate the schema: When we click Yes, the File / Save dialog opens. I saved the schema as divisions.xsd, and the mapping with the new XML Schema inserted looks like this: And the Properties dialog for the XML Schema component automatically contains the API /divisions URL as the Input XML File: Check the Work We want to filter the Groupon divisions data to build a list of id names to use for deal queries for each locality. But before we go any further, now might be a good time to apply the text file trick from the Quick Solution for Complicated Functions blog post to look at the id values. When we insert the text file and connect the divisions and id schema elements, the mapping looks like this: We connected the division element to Rows in the text file in order to generate a new row in the text file for each unique division, so that Field1 in each row will hold the id. Clicking the Output button now generates this result: All we need to do is apply the concat string function to build the list of /deal URLs for all division IDs. The next step in the mapping looks like this: Rolling the cursor over the constant connected to value1 of the concat function displays its full definition: When we click the Output button to execute the mapping, the Output file now looks like this: As a further review, we can open the generated XML Schema in XMLSpy and display it in graphical Schema View: So far we have:

built a MapForce mapping that queries the Groupon API for all divisions
extracted the division id fields
and built a list of URLs for API queries to get the deals in each division

In the next post in this series we will process the list of deal queries as the input for a new mapping component and filter the output for some interesting information. Find out for yourself how easy it is to apply MapForce to convert data from a Web API! Download a free 30-day trial of MapForce.
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: Altova XMLSpy, data integration, data mapping, MapForce, MissionKit, Web services, XML Schema

Altova Named to SD Times 100

June 15, 2010/in JSON /by Niki Devgood

the award for “Tools & Frameworks” goes to Altova!

We are honored to be named to the SD Times 100 for the sixth time! The award, now in its 7th year, recognizes the top innovators and leaders in multiple software development industry areas. Each year the editors of SD Times scour over thousands of nominations in search of companies, non-commercial organizations, open source projects, and other initiatives that exemplify innovation and leadership. When choosing the SD Times 100, the editors consider each company’s offerings and reputation. They listen to the “buzz” — how much attention a company, its products and technologies are creating — as a sign of leadership within the industry. Did the company set the industry agenda? Did a company’s products and services advance the software development art? Were programmers anxiously awaiting its developments? For the judges of the SD Times 100, these are qualities that mark a leader. So what makes us a SD Times 100 winner? In Version 2009, our MissionKit suite of XML, database, and UML tools added key support for working with industry standards like Extensible Business Reporting Language (XBRL) and Health Level 7 (HL7) to help companies adopt these standards while reducing costs and development time. We also rolled out over 70 new, customer-requested features in Version 2010 as part of our “Most Wanted” release, including support for WSDL 2.0, JavaScript Object Notation (JSON), and SysML, as well as compatibility with Windows 7 and 64-bit operating systems, and much more. Download a free, 30-day trial of the MissionKit (v2010r3) and see what’s new in the latest version (v2010r3) of our awarding winning suite of XML, database, and UML tools.

Tags: Altova, Award, MissionKit, SD Times 100

Visit Altova at DevConnections

November 6, 2009/in Events, JSON /by Cindy Neely

The Altova team is gearing up for Microsoft DevConnections 2009 next week! Beginning November 10, you can find us in booth #329 at the Mandalay Bay Convention Center in Las Vegas. Want to know more about our recently released "Most Wanted" Software Version 2010? Stop by the booth for a demo of all the latest product updates. We’ll be happy to show you support for additional technologies, such as WSDL 2.0 support in XMLSpy and MapForce, JSON editing and conversion in XMLSpy, and SysML support in UModel. The new version also delivers enhanced support for technologies like XBRL, offers a completely new design paradigm that gives StyleVision users a new option for creating stylesheets and electronic forms, and much more. Viva Las Vegas! Oh, and if you don’t happen to be attending DevConnections, you can catch us the following week at PDC in LA.

Tags: Altova events, Connections, DevConnections, v2010

XMLSpy’s Most Wanted

November 4, 2009/in JSON /by Erin Cavanaugh

Altova's Most Wanted When we announced that Altova Software Version 2010 included over 70 of the features most requested by our customers, we weren’t talking about little tweaks and enhancements, but major new functionality! To show you we mean business, I’ll outline some of the most requested features added to XMLSpy 2010 here.

WSDL 2.0 Support

In response to requests from from Web services developers, the graphical WSDL editor in XMLSpy 2010 now supports the latest version of the WSDL standard, WSDL 2.0. This adds to existing support for WSDL 1.1, giving you the choice of which version of the standard to work with. The WSDL editor automatically provides the correct editing environment for the version currently being utilized, and XMLSpy even provides one-step conversion capabilities for migration between WSDL 1.1 and 2.0. Since the XMLSpy WSDL editor uses a graphical interface (you can, of course, also work in Text View if you wish), you can easily visualize the structure of your WSDL document and edit it using drag-and-drop functionality and context-sensitive entry helpers, which offer the relevant choices based on the selected WSDL version.

WSDL 1.1/2.0 editor

Enhanced XBRL Functionality

Since we added support for XBRL validation and XBRL taxonomy editing in XMLSpy 2009, we’ve received excellent feedback from customers, including some feature requests that we were able to address in v2010. The new XBRL documentation generation capabilities of XMLSpy 2010 make it easy to generate comprehensive documentation – in RTF, MS Word, or HTML – for your XBRL taxonomies. Multiple options let you choose exactly what to include in the documentation, and the resulting output (snippet shown below) includes hyperlinked components for easy navigation. XBRL taxonomy documentation

Another option for documentation is to print the graphical representation of your taxonomy as it is shown in XMLSpy’s graphical XBRL view. XMLSpy 2010 also includes the new XBRL Taxonomy Wizard to give you a head start when creating a taxonomy. Simply enter the company name, ticker, or other identifier for your XBRL taxonomy, and then select the base taxonomy to extend (if any).

XBRL Taxonomy Wizard

XMLSpy creates the required taxonomy files and prompts you to select the entry points of the base taxonomy. Once you click finish, XMLSpy 2010 displays the newly created XBRL taxonomy files in the graphical XBRL Taxonomy Editor, where you can continue editing and refining the taxonomy in a visual manner. The new Find in XBRL and XBRL Sort options in XMLSpy 2010 meet customers’ requests for quick, easy ways to find data in and

navigate through large, complex XBRL taxonomies.

XBRL Taxonomy Editor

JSON Editor

We’ve recently heard from a lot of developers working on Web 2.0 and Web services apps in XMLSpy who also use JSON – so we decided to add a JSON editor in XMLSpy 2010. You can compose JSON strings in Text View or Grid View, and even convert between XML and JSON. In Text View, the JSON editor provides syntax coloring, line numbering, source folding, bookmarking, and more, making it easy to comprehend and navigate your JSON code, and find and edit strings. Intelligent JSON editing populates the Elements entry helper window with a dynamically built list of the elements present in your JSON file, which you can insert with a double-click.

JSON editor text view

Intelligent JSON editing is also available in Grid View, which provides graphical representation that shows the structure / outline of a JSON document through a set of nested containers. These can be easily expanded and collapsed to get a clear picture of the document’s tree structure, and drag-and-drop editing is supported.

JSON Editor Grid / Outline View A final must-have feature for working with JSON is the JSON <=> XML converter in XMLSpy 2010. One click lets you, for example, convert an XML file to JSON for transport with JavaScript, or convert data received in JSON format to valid XML. No more JSON vs. XML arguments – XMLSpy gives you the best of both worlds.

Redesigned Scripting Environment & Forms Editor

XMLSpy includes an integrated scripting environment and forms editor that has been redesigned for this latest release. Scripts can be written in JScript or VBScript to access and interact with the XMLSpy API, allowing you to modify and add functionality to your installation of XMLSpy 2010. Improvements and optimizations in Version 2010 include:

Access to most of the .NET framework
New form editor controls
Testing & debugging of macros directly in the scripting editor
Execution of macros directly through XMLSpy menus
Improved entry-helpers & auto-completion in the scripting editor

Read more about the “most wanted” features in XMLSpy and the rest of the Altova MissionKit. Please be sure to let us know your most wanted features, either by commenting here on the blog or entering a feature request.

Tags: JSON, v2010, WSDL, WSDL 2.0, XBRL, XML Editor, XMLSpy

Altova’s Most Wanted Edition – Version 2010 Announced

October 28, 2009/in JSON /by Erin Cavanaugh

Altova's Most Wanted

We are excited to have launched the MOST WANTED edition of the Altova MissionKit today! The MissionKit 2010 represents the most wanted functionality as reported by you! We’ve wrangled over 70 new features into this version of Altova’s XML, database, and UML product line to deliver on your list of demands. Check out the highlights for each product included in the MissionKit 2010 tool suite below; subsequent posts will cover the new features in each product in greater detail.

WSDL 2.0

One of the features most often asked for is support for the latest version of WSDL: WSDL 2.0. In addition to existing WSDL 1.1 editing and validation capabilities, XMLSpy now supports WSDL 2.0 in the graphical WSDL editor. It also allows one-click conversion between WSDL 1.1 and 2.0 files.

WSDL 2.0 support has also been added to version 2010 of the MapForce data mapping tool, allowing you to connect to WSDL 2.0 (or 1.1) Web services and integrate their functionality into mappings of XML, databases, flat file, EDI, Excel 2007, and XBRL data. You can also use MapForce to build new Web services based on WSDL 2.0 definitions. MapForce 2010 delivers a number of other enhancements for data mapping, conversion, and integration.

XBRL Enhancements

Several new features for working with XBRL have been added in version 2010. XMLSpy includes a new XBRL Taxonomy Wizard for getting a head start creating taxonomies in the graphical XBRL Taxonomy Editor. You can also now generate comprehensive documentation for your XBRL taxonomies, or simply print the graphical XBRL representation. Find and sort capabilities let you work more easily with large, complex taxonomies.

XBRL Taxonomy Editor

If you’re not familiar with the extensive support for working with XBRL in the Altova MissionKit, check out the XBRL Solutions Center page.

JSON Support

With more and more developers working with JSON in the XML, AJAX, and Web services apps they’ve created in XMLSpy, we’ve received a lot of inquires about JSON support. Now you can edit your JSON files directly in XMLSpy, using Text View or the graphical Grid / Tree View with intelligent JSON entry helpers. XMLSpy even supports one-click JSON <=> XML conversion. Check out the rest of the features most desired by XMLSpy users.

New Design Paradigm in StyleVision

The StyleVision graphical stylesheet design / single source publishing tool now gives you a new, flexible option for designing stylesheets and electronic forms. The new design paradigm lets you create templates within layout containers, and even optionally upload a blueprint image on which to base your design. This way, you can specify your design first and add XML and/or database content after.

Electronic Forms Design

This new form-based design option is made possible by new support for absolute positioning in design layouts. Instead of having content and layout elements flow on the page automatically, like a typical web page layout, you can now specify exact X and Y coordinates for each element to absolutely position it on a page. This feature gives you more control over form design in the manner of desktop publishing applications, but also adds the powerful single source publishing capabilities for XML, XBRL, and database data that have always been present in StyleVision. Read about the numerous other enhancements in StyleVision 2010. Authentic 2010 users will directly benefit from many of the new StyleVision features for true electronic forms design described above. The sophisticated e-Forms created in StyleVision 2010 are presented in Authentic’s WYSIWYG-interface for XML and database content editing by non-technical business users. In addition, with the release of Version 2010, Authentic is now offered in Enterprise and Community Editions, both of which are available as either as a desktop application or browser plug-in. The new Authentic 2010 Enterprise Edition requires a paid license and provides advanced features for WYSIWYG XML and database content editing. Authentic 2010 Browser Plug-in Enterprise Edition is offered on a 12 month license term basis. Authentic 2010 Community Edition is available under a free license and is also offered in desktop and browser plug-in versions.

SysML Support

Altova’s affordable UML modeling tool has become even more robust with support for SysML, which is related to UML but optimized for designing software to operate and control embedded systems and other complex devices. UModel 2010 supports all SysML v1.1 diagram types and elements with all the same productivity-enhancing features available for UML and BPNM modeling, including code generation in Java, C#, and Visual Basic.

SysML modeling tool

Read about all the new features in UModel 2010.

New Diff/Merge Capabilities

Both DatabaseSpy and DiffDog have received some exciting and oft-requested new diff/merge features in version 2010. DatabaseSpy, the highly-affordable, multi-database query, design, and comparison tool now allows you to compare and merge database schemas between databases of the same type or across different database types (all major relational databases are supported). DatabaseSpy displays differences between database schemas in a graphical manner, and you can generate a change script to merge changes in either direction.

Database schema diff/merge

DatabaseSpy 2010 also adds the ability to convert database structures between different database types, for example, to migrate from MySQL® to Oracle® or SQL Server®. Version 2010 of the DiffDog diff/merge tool for files, directories, and databases includes functionality similar to DatabaseSpy for comparing and merging database schemas, and it also provides the most wanted XML Schema diff feature. Since XML Schemas are ever-evolving as requirements change, existing XML files often also need to be updated to remain valid. You can now compare two XML Schemas graphically in DiffDog 2010, and then DiffDog will generate an XSLT file to transform any related instance documents according to the new schema. Alternatively, you can choose to generate an Altova MapForce mapping file based on the DiffDog comparison to further refine the transformation

between schemas.

XML Schema Diff
clip_image012

Collect your Handsome Reward

If you’re an Altova customer with active Support and Maintenance Package, simply download and install version 2010, and your existing key code will activate the new version. New customers can download a free, fully-functional 30-day trial to start working with all these new features today.

Tags: Altova