Tag Archive for: XMLSpy

XML Validation Speeds Reach New Levels


Let’s take a look at a few of the features recently added to the Altova MissionKit desktop developer tools and complete line of server software products, including screaming-fast XML validation speeds using RaptorXML Server inside XMLSpy,  frequently requested functionality for EDGAR filing validation, support for SQLite databases and the TRADACOMS EDI format, and much more. You can always learn about the latest and greatest new features on the Altova What’s New page.

 

KS110642_700x335
Read more…

Tags: , , , , , , , , , ,

Innovative New XPath/XQuery Tools


Support for XPath and XQuery in XMLSpy got a major boost with Release 3 of Version 2015. In addition to adding support for the latest standards versions – XPath 3.1 and XQuery 3.1 – we’ve added innovative new tools for developing and testing XPath and XQuery expressions in the XPath/XQuery window.

These new tools make it much faster to develop, evaluate, and refine expressions and statements before adding them to larger XSLT or XQuery documents. This will save you time and frustration – because you can understand how each operator or function works and ensure that your expressions return the desired results immediately. I found the function descriptions and entry helpers especially helpful for learning how the new maps and arrays features in XPath/XQuery 3.1 are implemented.

XPath and XQuery Tools

Read more…

Tags: , ,

New Support for XPath/XQuery 3.1, WS-Security, and More Debut in R3


Release 3 of the Version 2015 Altova MissionKit desktop developer tools and Altova server software products is now available with updated standards support, customer requested features, and innovative new functionality that will save you time on everything from XPath and XQuery development, to advanced data mapping, to XBRL report rendering and beyond. You can skip right to the good stuff and upgrade now, or click Read More below for all the details.
clip_image005

Read more…

Tags: , , , , , , , , , , ,

Automating Data Integration Workflows – Download the E-Book


We’ve published an e-book titled Automating Data Integration Workflows with Altova FlowForce Server that is now available for free download from the Altova Web site.

Here is a brief description: FlowForce Server is a highly-customizable platform to automate data transformations defined by MapForce data mappings, report and document generation, and other tasks on dedicated servers, virtual machines, or workstations scaled for the scope of the project. FlowForce Server empowers data architects, analysts, and other IT professionals to efficiently complete enterprise-level data integration tasks.

Automating Data Integration Workflows with Altova FlowForce Server provides an overview of the Altova FlowForce Server platform through a series of real-world data integration tasks and examples.

Automating Data Integration Workflows with Altova FlowForce Server

The new e-book collects updated information from the Altova Blog, the Altova Web site, and some all-new material, organized in one location to help new FlowForce Server users get started and to suggest new applications and efficiencies for experienced users.

Read more…

Tags: , , , , , ,

New XBRL Tools for Compliance and Beyond


It can be hard to keep up with all the exciting developments in the XBRL space. Standards are constantly evolving to make compliance easier and deepen the benefits provided by XBRL for internal analysis and reporting. As a leader in XBRL development and validation, Altova is committed to supporting the latest standards, coupling strict conformance with high-performance, easy to use tools. With the latest product launch, Version 2014 Release 2, we’ve added some important enhancements, including significant increases in validation speeds, XBRL Table Linkbase support, and extended XBRL Formula tools. Below is a portion of an XBRL Table generated in XMLSpy, using XBRL Table Linkbase to render revenue calculations in new and productive ways:

XBRL Table

Let’s take a look at the benefits of these new features for your XBRL implementation.

Read more…

Tags: , , , ,

DiffDog E-book Now Available


We’ve published an e-book in multiple formats for your favorite tablet or other e-reader.

DiffDog Field Trials e-book cover image

DiffDog Field Trials is a collection of information from the Altova blog, Tech Notes from the Library section of the Altova Web site, and DiffDog product feature pages. The e-book describes a wide range of real-world applications for DiffDog, Altova’s XML-aware diff/merge tool for files, folders, directories, and databases.

You can download DiffDog Field Trials from the Altova Web site in various formats:

· EPUB (1.57 mb)

· Kindle (3.08 mb)

· PDF (11.1 mb)

· iBooks (16.5 mb)

Or, look for DiffDog Field Trials soon in your favorite online electronic bookstore.

Editing EPUB files with DiffDog

As you can see from the list above, EPUB, the XML-based free and open e-book standard by the International Digital Publishing Forum (IDPF), is potentially the most efficient sharable file format. An EPUB document is based on a ZIP compressed archive containing style information, text formatted as html, and the images that appear in the e-book.

Altova XMLSpy provides powerful support for accessing, editing, validating and previewing EPUB 2.x documents. We used XMLSpy to assemble the EPUB version of our book, where we were able to interactively preview the result as we worked:
XMLSpy EPUB Preview Window Read more…

Tags: , , , , ,

XML Validation: The Need for Speed


When it comes to XML validation, more is better. By more, of course, I mean more speed, and more data. The faster you can get a list of validation errors – along with the information to help fix them – the better. Over the past few releases, we’ve made this an important priority in XMLSpy, first in Version 2013 with the introduction of Smart Fix XML validation, which actually provides a fix for a validation error that you can apply automatically.

In the latest release, we’ve improved on Smart Fix even further by enabling it in the graphical XML Schema editor, in addition to Text View and Grid View.

Smart Fix XML validator

Simply review the list of fixes automatically generated by XMLSpy, choose the one required, and click Fix.

You’ll also experience faster-than-ever XML validation speeds in XMLSpy 2014 thanks to integration of the super-fast RaptorXML engine, which also adds validation support for XML Schema 1.1, XSLT 3.0, and XQuery 3.0 (and earlier versions) with the strictest conformance to the corresponding standards.

Last but not least is the ability to display multiple validation errors at once, which has been requested by many customers.

Reporting multiple XML validation errors

You can now choose any number of errors to be reported during validation. As always, detailed information is returned for each validation error, allowing you to immediately visualize multiple issues and correct them in any order. The multiple tabs of the validation window let you store the list of errors as you re-validate, which can also be useful when fixing multiple issues.

If you haven’t upgraded to the latest version of XMLSpy, now’s the time. New customers can try all these features for free for 30-days.

By the way, if you’re running an older version of XMLSpy and want to see what’s been added since you last upgraded, the New Features Index is a great page see it all at a glance.

Tags: , , , ,

XML Wildcard Data Mapping and Transformation in MapForce


MapForce 2014 now supports mapping of XML wildcards that create a designed-in extension mechanism for XML Schemas, implemented as <xs:any> or <xs:anyAttribute>.

Consider this XML fragment viewed in XMLSpy:

Fragment of an XML document following the GPX XML Schema

This small portion of a very long file shows two track points recorded during a road trip by a Garmin GPS device. The XML data follows the GPX XML Schema, with Garmin extensions to add speed and course heading values. XML wildcard mapping lets you map these extensions in MapForce just like any other elements explicitly defined in the XML Schema.

Let’s assume we want to read the .gpx file and extract the highest speed recorded for a trip:

Output of the completed mapping Read more…

Tags: , , ,

Use XPath Expressions to Refine Data Selection


In this era of big data we often need to simplify analysis and communication by creating customized views into sections of a larger file. XPath, short for XML Path Language, is designed to allow users to address parts of large XML documents. XMLSpy supports XPath with an XPath Analyzer window and in its interface for charting XML data, MapForce supports XPath functions for XML data mapping, and StyleVision supports XPath in conditional templates, extension templates, and template filtering.

faces-laptop

Read more…

Tags: , , , , ,

FlowForce Server Supports RaptorXML


Altova FlowForce Server supports RaptorXML Server and RaptorXML+XBRL Server.

Log details of a RaptorXML job step

Altova RaptorXML is the third-generation, hyper-fast XML and XBRL processor from the makers of XMLSpy. RaptorXML is built from the ground up to be optimized for the latest standards and parallel computing environments. Now FlowForce Server jobs can include steps to validate XML, transform XML with XSLT, execute XPath and XQuery, and even perform complex XBRL operations for financial reporting including validating XBRL taxonomies, and validating XBRL instances against XBRL taxonomies with support for XBRL Dimensions and XBRL Formula, which define new, compatible functionality to extend XBRL 2.1.

For example, we can return to the FlowForce Server job described in Taming Bad Input Data with FlowForce Server and add a RaptorXML operation to validate the XML files created from raw GPS data captured by a digital camera.
Read more…

Tags: , ,

Editing, Converting, and Generating JSON


As the use of JSON as a data transport protocol increases, I thought it would be useful to take a look at JSON support in XMLSpy. There’s been much debate about advantages of JSON vs. XML, but when you boil it down, there are simply some cases for which JSON is the best choice, and others where XML makes more sense. This article on the XML Aficionado blog unpacks this topic quite well.
While you might need to choose between JSON and XML depending on the development task at hand, you don’t have to choose between code editors – XMLSpy supports both technologies and will even convert between the two. Let’s take a look at how that works.
Editing JSON
To make JSON editing as easy as possible, Altova extended its intelligent XML editing features to the JSON editor. If you choose to edit JSON in text view, XMLSpy helps you along with syntax coloring, bracket matching, source folding, entry helper windows and menus, and so on.

clip_image001

Here’s that same JSON file in grid view, which offers a graphical representation of the JSON structure with drag and drop editing. Both views provide JSON syntax checking and advanced error checking features.

clip_image002

Another common requirement is converting XML to/from JSON, which is a one-click option on the XMLSpy convert menu. The JSON data above has been converted to valid XML:

clip_image003

Having JSON editing and conversion functionality directly inside the XML editor is quite useful, especially if you’re using the free Eclipse or Visual Studio integration package.
Check it out and let us know what you think.

Tags: , ,

Expect the Unexpected – Altova MissionKit Solves a Number Format Mystery


Every time you receive data from an outside source, there is a chance it won’t arrive in the form you expect. This can require special accommodations for the rare and unlikely to make a real-world data mapping and transformation solution robust and reliable.

We processed literally dozens of .gpx files, containing hundreds of coordinates each, through the MapForce mapping we wrote about in the blog post Web Service as a Look-Up Table to Refine GPS Data. Then one day we ran a new file and encountered the error below, which caused the mapping to fail:

Error message during mapping of Web services data

Reaching into the Altova MissionKit to combine features of MapForce and XMLSpy, we quickly diagnosed the issue and developed a solution we can also reuse in future mapping projects.

Read more…

Tags: , , ,

The Altova MissionKit 2013 Introduces Seamless Integration of XMLSpy, MapForce, and StyleVision in Java Applications


The Altova MissionKit 2013, empowers developers to integrate XMLSpy, MapForce, or StyleVision functionality seamlessly in custom Java applications for Windows. This frequently-requested capability adds to existing support for integrating these MissionKit tools in Visual Basic or C# applications, giving developers flexibility to add some or all XMLSpy, MapForce, or StyleVision views and functionality to their own custom apps.

Version 2013 of XMLSpy, MapForce, and StyleVision include a new API that allows each to run inside a window within a Java application developed using the Java Abstract Window Toolkit (AWT) or Java Swing. This functionality allows MapForce 2013, for instance, to be embedded in larger applications where data mapping and transformation is only one requirement.

MapForce running in a Java ActiveX window

Altova provides sample applications with XMLSpy, MapForce, and StyleVision illustrating use of the Java API. You can access the sample applications from the command line or from within Eclipse.

Read more…

Tags: , , , , , , ,

Do You Believe in (XML) Magic?


One of the most exciting features in our recent Version 2013 release is Smart Fix XML validation in XMLSpy. Smart Fix is so magical it can revolutionize XML validation as we know it. While the XMLSpy validator has always been powerful, with new features added often, Smart Fix really delivers the “wow” factor: automatic error correction.

clip_image002 Let’s take a look at how it works.

Read more…

Tags: , , , ,

Have you ever wanted a schema for Apache Ant?


Recently I was having some issues with the Ant files for a software project I was working on. I was editing one of them in XMLSpy but wasn’t getting entry help or code completion because I didn’t have a schema assigned to my script.

That seemed like a simple problem, so off I went looking for the schema.

I quickly found a Stack Overflow question looking for the same thing. The answer turns out to be, there is no Ant schema or DTD. The reasoning is straightforward. Above the challenge of making a flexible and comprehensive model of the Ant core, it would be able to include all the external tasks.

I immediately remembered that I’d been here before a couple years back, but this time I figured I’d use XMLSpy’s schema generation feature to try to do something about it…

 

XMLSpy XML editor

Read more…

Tags: , , ,

New! XSLT Technology Training


We’re excited to introduce our new XSLT Technology training as the latest Altova Online Training offering. As with all our trainings, XSLT Technology is released as a free, self-paced course, available online, so students can fit it into their busy schedules.

XSLT transforms XML data into other formats, and this course will transform a beginner XML student into an advanced user. Intermediate and advanced students will gain valuable techniques to add to their XML toolkits.

clip_image002

Read more…

Tags: , , , , , , , , ,

Release 2 of the Altova MissionKit 2012


It’s an exciting week here at Altova! We have several new offerings to tell you about, including Release 2 of the Version 2012 Altova MissionKit and entire product line, a new Altova Online Training course on XSLT Technology, and a new service called Altova Product Consulting. clip_image001 Let’s start with what’s included in the latest product release. Version 2012 Release 2 delivers a host of the new features that have been most frequently requested by our customers, including support for EPUB 2.x, sorting of data mapping results, RichEdit functionality for Authentic eForms, code generation from UML sequence diagrams, and much more.

Version 2012 Release 2 Highlights

EPUB As the demand for e-books and other digital publications has exploded, so have the number of requests we’ve received from developers who need an easy way to create and edit these documents in XMLSpy 2012. In Release 2 we’ve added support for EPUB® (electronic publication), the open standard for e-books from the International Digital Publishing Forum (IDPF). It’s easy to view and work with the various files that make up an EPUB document in XMLSpy’s Archive View, which supports creating, editing, validating, and previewing EPUB digital publications. clip_image003 XMLSpy already provides validation and support for editing XML, XHTML, CSS, and other standards-based files that comprise EPUB documents, making it a natural choice for creating and working with EPUB content. Check out all the new features in XMLSpy. Sorting for Data Mapping Projects Another frequently requested feature is the ability to sort data in mapping projects where the final output format needs to be ordered differently than it was in the input. MapForce 2012r2 includes two powerful new sorting features that you can apply in a variety of ways. The screenshot below shows a new function that performs a multi-level sort on the source data before writing the output file: clip_image004 Another option is available for sorting relational input. MapForce users have long been able to sort database data through ORDER BY statements in SQL queries. However, SQL queries that operate inside the database are not always sufficient for complex data mapping tasks. MapForce 2012r2 adds additional database sorting functionality to the existing MapForce SQL-WHERE component to become SQL-WHERE/ORDER, which is especially useful when database input requires additional processing that cannot be accomplished in a SQL query, or other data or conditions can impact the sorted order of data rows. clip_image005 Read more about new MapForce features. Other important features added in R2 include RichEdit functionality for Authentic eForms created in StyleVision, code generation from UML sequence diagrams in UModel, support for logical files in IBM® iSeries® databases across the product line, and much more.

Upgrade Info

Check out the rest of the features added in Release 2. This new version is free to download and install for customers with Support and Maintenance. If you’re not already an Altova customer, you can download a free, fully functional 30-day trial.

XSLT Technology Training

We are thrilled to announce the latest Altova Online Training course, XSLT Technology. This free, online, self-paced class introduces students to the concepts of XSL, XSLT, XPath, XSL-FO, and more, through guided tutorials and videos. You can practice working with these technologies using the tools of the Altova MissionKit to transform XML to XML, to HTML, and to PDF. clip_image007

Altova Product Consulting

Have you ever wished you could purchase a few hours of a product expert’s time – whether you need to determine if a particular tool could solve your problem, or need help creating a schema or mapping project, or simply require in-depth product support? We’ve heard from a lot of customers with this very requirement – and in response we have created Altova Product Consulting. Altova’s Product Consulting engineers are experts on Altova software and related technologies. Services include determining if an Altova product meet the needs of a particular project, creating a MapForce mapping file (.mfd), designing a StyleVision design (.sps), and so on. Unlike typical complicated consulting engagements, Altova Product Consulting services are sold per hour, the allotment of which is agreed upon in advance, so there are no surprises. Get more information on specific services provided, pricing, and how to get started.

Tags: , , , , , ,

XML Editor Supports HTML5 and CSS3


The Altova MissionKit Web tools received an important update with our recent Version 2012 release: support for HTML5 and CSS3. You’ll find support updated to include the most recent versions of these Web standards in both XMLSpy 2012 (and higher) for code editing and StyleVision 2012 for graphical stylesheet and report design.
clip_image001
Let’s focus on the HTML5 and CSS3 editors in XMLSpy here – StyleVision functionality will be covered in a future article.

HTML5 Editor

XMLSpy has long supported editing of HTML4 and, of course, XHTML – and now those intelligent coding features extend to HTML5.
If you’re already an XMLSpy user, you’ll immediately recognize the helpful syntax coloring, source folding, and line numbering in advanced Text View when you open your HTML5 file for editing. As you type, you’re presented with valid HTML5 element and attribute choices in a drop down menu as well as in static entry helper windows, so you can complete code in your preferred working style. At the same time, code completion helps you work faster while ensuring elements are closed properly.
clip_image002
These intelligent editing features are applicable to the version of HTML you’re working with – when you open an existing HTML5 document or start a new file from scratch, all the HTML5 elements and attributes are immediately available in the entry helper windows and drop down menu. Some of the most notable new elements in HTML5 include:

  • <video> and <audio> for media playback
  • <canvas> for dynamic rendering of 2D images
  • <article>, <section>, <header>, <footer>, and <nav> for including richer semantic information to describe content
  • <calendar>, <date>, <time>, and other form control elements
  • And so on

XMLSpy includes an integrated Browser View that lets you see the results of your HTML5 coding immediately. The Browser View renders your page using your installation of Internet Explorer directly in XMLSpy…
clip_image004
…but also allows you to immediately view your HTML5 Web page in an instance of any browser you have installed on your machine.
clip_image005
This allows for quick testing and debugging to ensure cross-browser compatibility.
To get an idea of how some of the new HTML5 features look in action, check out the simple HTML5 example document that is supplied in the XMLSpy 2012 Examples folder and linked from the Example project. To learn more about HTML, there are many excellent resources on the Web, including  http://www.w3.org/TR/html5/.

CSS3 Editor

Working hand-in-hand with the HTML5 editor is the XMLSpy CSS3 editor, which also provides syntax coloring and entry helpers, as well as bookmarking and source folding in Text View.
clip_image006
A CSS Outline window displays an outline of the document organized by its selectors listed in groups. Each group can be collapsed and expanded, and clicking a selector in the CSS Outline highlights it in the document. CSS Properties and HTML elements entry helpers are shown in windows as well as context sensitive drop-down menus, and XMLSpy even includes screen tips that provide a definition of each property and its possible values as you type.
clip_image007

You can take advantage of XMLSpy’s integrated Browser View here, too, to see the results of your CSS edits immediately in a linked HTML page.

clip_image008

Why XMLSpy for HTML?

Sure, XMLSpy is known for all things XML, but it’s also a clear choice for HTML development. HTML5 and CSS3 editing in XMLSpy is facilitated by support for related technologies including XML, XSLT, WSDL, and others, as well as integrated project management functionality, support for popular version/source control systems, and Microsoft® SharePoint® Server integration – giving you all the tools you need to develop the next generation of Web sites and apps.

We’d love to hear how you’re using HTML5 now – please drop us a line by leaving a comment below.

Tags: , , , ,

Processing the Groupon API – Epilogue


Rare edge cases can derail loosely coupled data mapping applications. This is especially true when you are consuming large datasets available over the Internet and have little or no influence over the source data. In this article we describe a debugging technique that lets developers working on data mapping and transformation projects quickly identify and accommodate unexpected data in a stream from a remote source. The Problem Last summer we wrote a series of blog posts describing how to work with the Groupon API to retrieve a subset of offers in all Groupon cities and format the list for a web browser or mobile device. MapForce output from the Groupn API, displayed on a mobile device We concluded with a command line to run a MapForce data mapping that calls the Groupon API over 150 times — once for each Groupon city, then filters the data to extract deals sold on the Internet instead of a physical location, and formats the results in HTML using StyleVision. Every morning we run the command line in a batch file that saves the HTML output on a local server so our colleagues can check it out with any Web browser to find interesting offers from all over the country. The mapping ran fine for more than two months until one day it failed with this error message: “Source-value “” of type dateTime could not be converted into target-type dateTime.” The specific explanation is that somewhere in the mapping where we expected a dateTime, we received an empty value. On a more abstract level, the error suggests a potential defect in the logic of our mapping strategy. Every time we call the Groupon API we receive a well-formed XML data stream enclosed in a <response> element, but the API specs do not include an XML Schema defining the data that may be returned. When we developed our mapping we needed to analyze the raw data and select the output we wanted, so our first step was to call the API to capture all the Groupon deals for one large metro area. We assumed we would get a large enough data sample to include every possible option in the API response. After our mapping ran successfully for two months, the API finally delivered a rare edge case that did not fit the pattern we expected. Debugging Tools MapForce provides debugging help. We can run our data mapping using the MapForce built in execution engine to see more details in the Messages window. MapForce Messages window siplays data mapping error The lines labeled Related location are hyperlinked back to components in the mapping where the error occurred. Clicking on the result error takes us to a format-dateTime function. format-dateTime function in MapForce We can either click the “” error or trace the value connector to identify the input element to the format-dateTime function. Either way, we locate the element that triggered the error. clip_image004 The suspect element resides in the input component that captures all the data returned by our calls to the Groupon API before any filtering or conversion takes place. When we designed the mapping, the endAt element in our sample data always reported the ending date and time for each Groupon offer, but for some reason we must have received an empty value in this field. If the error had occurred by running a local input file we could simply examine the file contents, but in this case the data came from multiple URLs, and is only held temporarily until it is mapped to the output component. Fortunately, we can apply a trick to easily modify the mapping and preserve all data received from the Groupon API. We simply copy the input component and paste a duplicate into the mapping. We can connect the response element from the original to the duplicate, which simultaneously maps all the child elements between the components. clip_image005 Our original input component is now connected to two output components. We can select which output component will be generated by the MapForce built-in execution engine by clicking the eye icon at the top right corner of any output component. The new output component simply saves a copy of everything in the input component. When we examine the raw data using XMLSpy, sure enough we find an empty element where we expected a date and time: clip_image006 The Solution Now that we know an offer might have no specific end time, we can plan for that possibility in the mapping. In the revised treatment of the endAt element, we do an if-test before the original format-dateTime function and provide an alternate output when the endAt element is empty. clip_image007 We had to work fast because all Groupon data is time sensitive. The edge case would eventually expire and disappear from the data stream. This experience showed us how important it is to have powerful debugging tools and to use them creatively, even after you think a data mapping project is running successfully! Altova MapForce is available in a free trial – the next edge case you solve could be your own. Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: , , , ,

Digging deeper with the Twitter API: iPhone 4S vs. Galaxy Nexus


We found some interesting data when we dug below the surface of the iPhone 4S vs. Galaxy Nexus debate using the Twitter Search API.In today’s world there is a vast quantity of data available online that can be used for research, market analysis, and competitive intelligence. While “Big Data” can be a problem for those who produce it, store it, and compile it, it is highly beneficial for those of us who are looking for answers.Some of that data is fortunately available to be queried online, and, in particular, there is a vast quantity of data on social media interactions out there.TweetsQueryingSearchAPIIn this article we will explore how to use the Twitter Search API from MapForce, Altova’s data mapping/conversion/integration tool, to aggregate data on recent user submissions (“tweets”) on two highly popular topics – the Apple “iPhone 4S” vs. the “Galaxy Nexus” as the latest hot Android phone – and extract some statistical data about the users engaged in those discussions. One of the benefits of this abundance of data available to us today is that we can query it in interesting ways and extract new meaning from it. While there are undoubtedly many existing services that already provide trends over Twitter topics (e.g., Trendistic), those services only offer very simple trends and do not allow us to query any deeper.But all of the underlying data is available for grabs if you are just willing to learn a tiny bit about web service APIs and how to use them to extract XML data for further processing. As a starting point, let’s use the Twitter Search API to query the stream of recent tweets for the last 100 postings that are about the “Galaxy Nexus”. The Usage Guidelines for Twitter Search tell us that using both words in a query will result in the use of the default operator, which is AND, so we are going to search for posts that contain “Galaxy AND Nexus”. So let’s try that and request the most recent 100 items:

http://search.twitter.com/search.atom?q=galaxy+nexus&rpp=100

If you follow this link, you will get a second window with a lot of raw XML data that is formatted according to the Atom Syndication Format specifications. Alternatively, you could request the data in JSON format, if you wanted to directly process it via JavaScript code by hand, but we will use the XML-based Atom format so that we can easily analyze the data and extract the information we want.Viewing the above search result in a browser is not very user-friendly, so we can take a quick peek at the XML data in our favorite XML Editor using the Open from URL function:TweetsAtomGridAs you can see, the data for each entry includes a language code, so for this example we will extract data from this Twitter feed as well as from a second search result on the “iPhone 4S” and combine them into one intermediate XML file for further analysis.Extracting XML data is really easy in MapForce: using the “Insert XML File” option to drop in an XML source, we can again specify the same URL as before. If needed, MapForce will automatically create an XML Schema for the supplied data so we can visualize it and extract information from it:TweetAtomMappingIn our mapping we have dropped in two sources on the left side – one using a query string to search for “Galaxy Nexus” and the other to search for “iPhone 4S” – and on the right side we have dropped in a simple XML Schema that will allow us to aggregate our data and analyze it more conveniently going forward. In this case the mapping between the two sides is straight-forward as we are only extracting basic information about the user, the date, and the language of the tweet, but in other applications the mapping could be more complicated and include functions as well as queries to other data sources, databases, or web services…Previewing the resulting XML data can be done directly inside MapForce using the output tab, and this is what we see as a result of our data transformation:TweetsRawDataNow we can easily use the reporting capabilities of StyleVision to group this data by language within each topic and count the number of posts in each language. We can then report this data in the form of pie charts, which produces the following interesting results:TweetsByLanguageObviously, this data is highly dependent on the date of execution and time of day, as well as the particular announcements happening about these products, so the numbers will fluctuate quite a bit, but it can be used as a nice monitoring for seeing different language-specific trends. And once this has been set up, the report can be refreshed easily with the click of a button to get a snapshot at that point in time. For more long-term analysis it would of course be necessary to modify the mapping a bit to query more than 100 recent tweets.In this article we have used Twitter’s Search API as one example data source and only looked at language as one unique data point, but there are many more interesting sources of data available online today, and this approach can be used on all of them in a similar fashion.If you want to experiment with other data sources and other kinds of information that you want to extract, we invite you to try for yourself. A free 30-day evaluation version of MapForce is available, and there are no limits on how you can use the other features of Altova’s data mapping and conversion tool for data processing tasks that go beyond analyzing social media trends…

Tags: , , , ,

Case Study: Altova Customer Succeeds with XBRL


XBRL is mandated for most public companies. So why are private organizations and non-profits jumping on the bandwagon? This case study examines a real-world success story. clip_image002   We were really excited when the folks at MACPA told us about their success working with XBRL. They set out to discover if XBRL could be used successfully (without a huge upfront investment) by small businesses and NPOs and ended up confirming not only that, but realizing benefits to their internal financial processes, as well.

Toward Ubiquitous XBRL

With close to 10,000 members, the Maryland Association of Certified Public Accountants (MACPA) is often looked to for their expertise on issues relevant to the field of accounting. The US Securities and Exchange Commission’s (SEC) mandate that public companies submit financial data in XBRL is one of those issues. Despite the potential of XBRL for reducing costs and increasing efficiency, many organizations are concerned about the time and expense that will be required to convert all of their financial data into XBRL, a process that can be further complicated when financial data is housed in multiple systems. MACPA set out to prove that these obstacles are easily surmountable: with the right tools, it’s possible to bring XBRL transformation in-house to not only comply with mandates, but realize greater efficiencies and transparency in various scenarios. In the process they discovered that tagging data in XBRL is valuable to private entities and non-profits as well as public companies facing a mandate. They took advantage of widely available XBRL software tools including the Altova MissionKit, which interfaces with multiple relational databases for XBRL mapping, tagging, and reporting.   clip_image003   In the end, the project turned MACPA’s financial data into a force for driving efficiencies and accountability. Once their internal accounting data was mapped to XBRL, they were able to automate burdensome data collection, transformation, and analysis tasks to gain more insight into their financial data. For instance, MACPA used their XBRL data to populate their financial Key Performance Indicator (KPI) system, significantly reducing the amount of time and effort required to prepare the KPI documentation. This in turn enables them to run the system at more frequent intervals. They are also now able to automate previously onerous tax filing tasks by mapping the association’s financial data in XBRL to the 990 tax return. (With almost 1.5 million exempt organizations in the US filing hundreds of thousands of Form 990s each year, the efficiency gained by using XBRL could be significant.)

“Ubiquitous XBRL could do for accounting/taxation what barcodes did for retail.” – Skip Falatko, MACPA Director of Finance and Administration

This project not only enabled MACPA to learn about XBRL and advise their members, but also to automate and enhance the way they dealt with their own financial data. And utilizing affordable tools like the Altova MissionKit confirmed that handling XBRL in-house is the way to go.

“Why outsource tagging [your data in XBRL]? If you tag it in house, then you own the data and can use it in myriad different ways as a productivity tool.” – Tom Hood, MACPA CEO and Executive Director  

Check out the complete case study to learn how MACPA brought XBRL transformation in-house to effect changes in efficiency and transparency. If you’re an accounting or technical professional who needs to learn more about XBRL, Altova offers free, self-paced online training and an educational XBRL whitepaper.

Tags: , , ,

XML Development with Database Integration


Did you know that XMLSpy connects to relational databases? One of the most compelling features of the Altova MissionKit is that numerous tools in the suite include offer deep integration with relational databases, providing seamless access to back end data for bi-directional conversion, integration, analysis, and reporting. image Let’s take a look at what you can do when you connect XMLSpy to your databases. Other database-enabled MissionKit tools will be covered in subsequent posts.All popular relational databases are supported in XMLSpy:

  • Microsoft® SQL Server® 2000, 2005, 2008
  • IBM DB2® 8, 9
  • Oracle® 9i, 10g, 11g
  • Sybase® 12
  • MySQL® 4, 5
  • PostgreSQL 8
  • Microsoft Access 2003, 2007

First step: Connect to and query the database

When you select Query Database from the DB menu, XMLSpy helps you connect to your database with the step-by-step Database Connection Wizard. Then, the DB Query window makes it easy to explore and/or edit data in the database you’re working with, either by opening existing SQL files or creating SQL scripts from scratch using drag-and-drop and auto-complete functionality. Once you execute your query, you can edit the database data in the results window, review changed fields (highlighted in pink), and commit the changes back to the database. Querying a database in XMLSpy

Next: Convert between XML and databases

Another common requirement is converting between XML and database models, and XMLSpy supports this in both directions. You can easily export database data to XML. If no schema is required, you can simply export the data to XML in its basic tabular format. Or, you can use the Create XML Schema from DB Structure option first, then import database data maintaining all the relationships and dependencies defined in the content model. Numerous options are available to specify the format of the schema, whether columns should be imported as elements or attributes, and the database constraints that should be generated in the XML Schema. Get schema from DB data Or, to go in the other direction, it’s just as easy to go from XML to a relational model in XMLSpy. The Export to Database dialog (accessed via the Convert menu) allows you to specify where to start the export, how to handle export fields, and which elements to include. Then, the data is instantly converted and stored in your database. image For times when you want to define a database with the same rules as an existing XML Schema, the Create DB Structure from XML Schema dialog lets you do so with numerous options. Any identity constraints included in the schema will automatically transfer to the database structure. Alternatively, it’s easy to define relationships between elements manually. Learn more about all these features for working with XML and databases in XMLSpy, or check out all the database tools available in the MissionKit.

Tags: , , ,

Meet the Altova Team at Oracle OpenWorld


clip_image002Some people say early October brings the most enjoyable weather to San Francisco, and Oracle OpenWorld and JavaOne will make things even better this year from Oct. 2 – 6. If you’re traveling to the City by the Bay for either event, plan to visit the Altova team at booth 106 in the Moscone South Oracle OpenWorld exhibit hall. We’ll be demonstrating all the tools in the Altova MissionKit with special emphasis on multi-database support, tools for working with XML in databases, and solutions for compliance with XBRL, NIEM, HL7, EDI, QR codes, and many other industry standards. Here’s your chance for a personal tour of that Altova tool or feature you’ve been curious to see – whether it’s advanced charting in StyleVision, Java code generation from UML in UModel, or refactoring XML Schemas in XMLSpy. We always enjoy meeting users face to face and hearing about the projects you’re working on too! You can also enter our Altova product raffle for a chance to win a license for Altova MapForce 2011 Basic Edition. We hope to see you there! clip_image003

Tags: , , , , , , , , ,

Processing the Groupon API with MapForce – Part 2


In Part 1 of this series we described how to connect Altova MapForce to the Groupon API. We queried the API for a list of Groupon divisions, then used the list to create API queries for all the current deals from every division. In this part, we will execute the /deals queries and filter the response for the most interesting data. The list of /deals queries we built previously looks like this: List of Groupon /deals queries generated by Altova MapForce To process all the queries, we can connect the list as a dynamic file input to a new mapping component. When we needed a new component last time, we dropped an API /divisions query into the mapping, and let MapForce create an XML Schema automatically. We could do the same thing here by dropping in an API /deals query as an XML input file. There’s just one small issue — although the Groupon API online documentation clearly describes the queries we can make, it is vague about the information that will be returned. Before we send dozens of queries to the API for all the current deals, we probably want to know a little more about the data that will come back.

Let’s Make a Deal Like Yogi Berra said, you can observe a lot just by looking. Let’s start by running a /deals query in XMLSpy. That will let us examine the response to a query for one division before we pull in a potentially unwieldy volume of data. The XMLSpy File / Open menu includes the same Switch to URL option we used in MapForce in the earlier post. If we enter the /deals API query for a division that covers a large metro area – say Dallas – we are likely to get enough deals instances to extrapolate the characteristics of the entire data set. XMLSpy opens the response to the /deals API query in Text view just as if we opened a local file: Example from the response to a Groupon /deals query, shown in XMLSpy As expected, we got quite a bit of data when we requested all the deals for a single division! A fast way to analyze the structure of this data is to use the XMLSpy DTD / Schema menu option to generate an .xsd file from the xml. Shown below is a reduced view of the entire generated .xsd file based on the response to the /deals query for Dallas: An xsd file generated by XMLSpy from the Groupon query We can dig even deeper, following Yogi’s advice like déjà vu all over again. Expanding all the elements to review the XML Schema reveals some curious anomalies. For instance, there are two elements named redemptionLocation with different definitions. The first contains a sequence of child elements: First use of the remdemptionLocation element And the second is defined as a simple string: Second use of the remdemptionLocation element Going back into the xml data for Dallas and searching for redemptionLocation displays these examples: One example of redemptionLocation in the body of the response And: One example of redemptionLocation in the body of the response And: One example of redemptionLocation in the body of the response Now this is really interesting, because redemptionLocation = ”online” identifies deals that can be redeemed from anywhere, instead of by a visit to a bricks and mortar location in the division where they are advertised. What if we ran the /deals API queries for all divisions and extracted a list of all the online deals? That would be one extreme Groupon! Only Ask for What You Need The Groupon /deals API query supports an optional parameter called &show= that allows users to limit the data returned. Applying this parameter can save bandwidth and reduce processing time for the data transformation by removing unwanted data from the API response. We can also simplify our final result by including only the most interesting information, including the link to the Groupon web page for each deal. After we remove unwanted elements from the generated Dallas schema, our final version for the summary of online deals looks like this: XMLSpy Schema diagram of the simplified Groupon xsd file When we add the &show= parameter to our MapForce mapping to request only the elements included in the simplified XML Schema, the queries look like this: Modified list of queries with the &show= parameter Now we can drop the revised .xsd file into the mapping and connect the list of API /deals queries as dynamic input. We don’t need to delete the text file we used to collect the list of queries — that might continue to be helpful for future debugging. Mapforce dynamic input file mapping These changes complete the input side of the data mapping. Defining the Data Transformation Output Back in XMLSpy we can make a couple more revisions to the input XML Schema to design a new version for output: XMLSpy schema diagram of the output file xsd We discarded the response element since it doesn’t add any value, and eliminated the redemptionLocation element that we don’t intend to include in the output. We also added a date element for a timestamp, because our output file will be a snapshot of data that is constantly changing. After saving this version of the .xsd file in XMLSpy, we can drop it into the MapForce mapping. Shown below is the output side of the mapping with the output component partially connected. The filter at the top reads the redemptionLocation element to select only online deals and the now function inserts the date: Partial view of the MapForce output file mapping The last revision we made in the output XML Schema was to change several element types from dateTime, Boolean, and integer to the string data type to allow more descriptive text Here is the complete definition of the mapping with the final connections to the output component: Mapforce data mapping for the Groupon API Now for the Payoff When we click the Output button MapForce processes the entire mapping from beginning to end using the MapForce Built-in execution engine. Here’s a breakdown of the steps:

  • Run the /divisions query to get the current list of divisions
  • Concatenate strings to build the list of /deals queries for all divisions
  • Run the /deals queries to create dynamic data for the input component
  • Filter for online deals to generate the output component, execute the remaining mapping functions, and add the timestamp after all the deals are processed

MapForce takes only a few seconds to complete all those steps and generate an output file with a series of deals that look like this: Output data from the MapForce mapping for the Groupon API In part 3 of this series we’ll design a stylesheet to automatically transform the XML output of our mapping into html for attractive presentation in a web browser and on mobile devices. See ya at the ballpark, Yogi! XMLSpy and MapForce are available together in the specially priced Altova MissionKit. See for yourself how easy it is to use the MissionKit to convert data from a Web API — download a free 30-day trial!
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: , , , , , , ,

Use Built-In XPath Functions


In developing one of the Altova Online Training courses, I sorted a list of books by the authors. I realized that my author field was a string of the author’s full name, so the books were sorted by the first letter of the string, or the author’s first name. It did not fit into the course to fix the sorting, but you can easily extract the last name from a string and use it for the sorting key using XPath functions. If you then use the books’ titles for a secondary sort key, you run into an issue with titles that start with “A”, “An”, or “The”. I want to use the title for the secondary sort key, but ignore a leading definite or indefinite article.Output the book list with a  sort corrected using XPath expressions Let’s take a look at how we created this XSLT code.

This article was written using XMLSpy as the platform, but the same XPath expressions can be used inside MapForce or StyleVision to achieve similar results. We can start with a simple XML book list. We have 4 books with author and title nodes. List of three books An XSLT to create a list of the books would look like this: Output the book list without a sort This will generate the following output: Unsorted Book List The books are output in the order they appear in the original data file. If we add xsl:sort to the xsl:for-each loop, we can arrange our output in other ways. Output the book list with a basic sort This will generate a sorted list, but not sorted properly. Output from XSL with Basic Sort Sorting author as a string, results in “Jules Verne” appearing ahead of “Mark Twain”. Also, “A Connecticut Yankee in King Arthur’s Court” appears ahead of “Adventures of Huckleberry Finn”. We want to ignore the indefinite article, “A”, so that “Adventures of Huckleberry Finn” appears ahead of “A Connecticut Yankee in King Arthur’s Court”. We can use XPath expressions to extract the sorting keys we want. Output the book list with a  sort corrected using XPath expressions Let’s examine the code before we look at the output. We replace “author” with “reverse(tokenize(author, ‘ ‘))[1]”. Tokenize breaks the author string into tokens using a single white space as the break point. So, “Jules Verne” is tokenized into “Jules” and “Verne”. Reverse reverses the order of the tokens to “Verne” and “Jules”. The one in square brackets chooses the first item in the list, “Verne”. This is the value that is used in for the xsl:sort function to arrange the books. This is not the perfect solution, but it works in our case. The title looks convoluted, but the logic is straightforward. The “tokenize(title,’ ‘)[1]” expression extracts the first word of the title. So, the first if test is “Is the first word of the title the word “A”? “. If it is, then we return the substring of the title that starts with its third letter, thus eliminating “A” and the space. If the first word of the title is not “A”, then we need to test it again to see if the first word of the title is “The”. If it is, we use the substring of the title starting with its fifth character, thus eliminating “The” and a space. If we fail both tests, then we just pass the title along as the sorting key. We could add another test to our code to see if the first word is “An”, but it is not needed for this data set. Executing this last XSLT, we get the following output. Output from XSL with Corrected Sort “Mark Twain” is now ahead of “Jules Verne”. “Adventures of Huckleberry Finn” appears ahead of “The Celebrated Jumping Frog of Calaveras County” and “A Connecticut Yankee in King Arthur’s Court”. The flaw in our approach to the author string is that we want “Jules Verne” to be treated as “Verne, Jules” for the sort, so that if we had a book by “Jimmy Verne”, the sort would treat them as different authors. Our code does not. Using “concat(reverse(tokenize(author, ‘ ‘))[1], reverse(tokenize(author, ‘ ‘))[2])” would sort “Jules Verne” and “Jimmy Verne” correctly, but this solution only will work with 2 word names. If an author had a suffix (“Martin Luther King, Jr.”) or multiple words (“George Herbert Walker Bush”), the code would fail. There are many exceptions to the general rules on alphabetizing names, and the code to allow for all variants goes far beyond the scope of this article. What we wanted to show was the ability to manipulate XML data on the fly using XPath expressions. We do not always have complete control on the format of our data sources, but using the power of XPath expressions, we can transform the data into the format that we need. A copy of the files used in these examples is available here.

Tags: , , , , , , , ,

Release 3 of Altova Software Version 2011


  New in Altova MissionKit 2011r3 PXF (Portable XML Form), XML digital signatures, and HIPAA data mapping are just a few of the many exciting new features we’ve introduced today with Release 3 of the Altova MissionKit 2011 and complete product line. Let’s take a look at some of those now. First…what on earth is a PXF?

What is PXF?

PXF is a new file format that – finally – makes editable XML forms truly portable. PXF files are configured in StyleVision 2011r3, where a developer creates a design that generates an interactive electronic form that can then be accessed and edited by non-technical business users in Altova Authentic (or in the Authentic View in XMLSpy). When the StyleVision design is saved as a PXF, the XML Schema(s), XML instance(s), electronic form, etc., are all embedded in the PXF file. The PXF can even optionally include the XSLT files autogenerated by StyleVision that allow the business user to publish his data in HTML, Word, RTF, and/or PDF – with a single click. After a PXF file has been created, it can be transported, downloaded, copied, and saved like any other data file. Portable XML Form Why PXF? The benefit of the PXF file format is that all the components required for editing XML content – and for the generation of output reports from Authentic – can be conveniently distributed in a single file. And, since Authentic Community Edition is a free product, PXF deployment couldn’t be more affordable. Authentic interface There are countless usage scenarios for the PXF file, but one example is to allow a business user to easily get important data into a valid XML document and send it back to the requester or to another department, such as HR or Accounting. The user receives a PXF file by email and opens it in Authentic. The XML file will be displayed in Authentic using the embedded StyleVision eForm design, and can be edited using the word processor-like Authentic interface with access to advanced interactive options, context sensitive entry-helpers, business logic validation, and more. The File | Save command saves changes to the PXF, i.e., the embedded XML is modified and saved. The File | Send by E-mail command makes it easy to send the updated PXF on to another user with a single click. In addition, when XSLT files associated with the SPS are included in the PXF, Authentic toolbar buttons let the user render his data in multiple formats for publishing or further communication. PXF takes the headache out of getting critical business data into XML – without sacrificing any of the benefits that XML brings to data integration and extensibility.

XML Digital Signatures

To address the growing need for security around XML transmissions, v2011r3 includes support for assigning and verifying XML digital signatures through the implementation of W3C XML Signature technology across multiple MissionKit tools. An XML Signature enables digital authentication for XML transactions by checking the integrity (whether the data has changed since it was signed) and the authenticity of origin (the identity of the signer). In contrast to other digital signature methods, XML Signature is uniquely suited to working with XML data because it includes measures for canonicalization, which involves signing the important data while ignoring inconsequential changes such as whitespace and line endings. In XMLSpy 2011r3, it’s easy to add an enveloped, enveloping, or detached signature to your XML-based files using either certificate e or password-based authentication. Create XML digital signature You can also verify signature(s) on files received. If the file changed at all since it was signed, verification will fail. Verify XML Signature Other MissionKit tools with XML digital signature support include:

  • MapForce 2011r3: When your data mapping project has XML or XBRL output, you can create an XML digital signature that will be added to output file produced when the mapping is executed.
  • StyleVision 2011r3: StyleVision designers can configure Authentic eForm output to allow XML instance files to support XML signatures.
  • Authentic 2011r3: When an Authentic eForm designed in StyleVision has XML signatures enabled, the Authentic user can sign XML data using an enveloped or detached signature as well as verify any XML signatures present when documents are loaded.

HIPAA Data Mapping

In addition to new features for Excel® data mapping and other enhancements, MapForce 2011r3 now includes native support for mapping HIPAA 5010 data. Health care enterprises that send or receive HIPAA information will be able to apply MapForce 2011 Release 3 to meet a number of requirements. Users of legacy healthcare IT systems that do not store data files internally in a format compliant with the latest HIPAA standard can use MapForce 2011r3 to map incoming HIPAA 5010 transactions to the enterprise internal format or database. Or, a MapForce mapping can be designed to generate HIPAA-compliant transactions for output from existing non-compliant data. If the enterprise wants to translate legacy healthcare data for internal storage in HIPAA 5010 format, MapForce 2011 Release 3 is the tool for that one-time transformation too! HIPAA EDI mapping Native HIPAA support adds to current support for HL7 (Health Level 7) and other EDI standards. Check out all the features added in the latest Altova release, and stay tuned for more details here on the blog.

Tags: , ,

Creating XML from Relational Databases


Sometimes following an example someone else created is a good way to get a quick start on a project. The downside is you might miss a better, more efficient solution. In our recent post on XML in the Cloud, we used DatabaseSpy to connect to a local MySQL database and to the Amazon Relational Database Service in the cloud. We used the Concat( ) function in a SQL SELECT statement to create XML formatted output from non-XML data as shown below. DatabaseSpy SQL query and result Our SELECT statement was based on an example in the MySQL documentation on XML support. Let’s take a little deeper look at the problem this statement tries to address. You can copy a DatabaseSpy Results table like the one displayed above and paste it into an editing window in XMLSpy, but the Results table alone does not create a well-formed XML document. To be well formed according to the W3C definition, an XML document must contain a root element. All other elements and logical structures must nest within the root. You can also think of the root element as a wrapper around the entire XML content, the same way the element <city></city> encloses each line in our original results. A Better Way to Create XML from Relational Data We don’t need to manually edit the results to add a root element, nor do we need to adapt our already-complicated SQL query to add the root. DatabaseSpy lets us easily export well-formed XML documents from database tables that contain ordinary data like our cities table. In the DatabaseSpy Export dialog we can choose XML Structure as the output format, click the cities table to select it from the database hierarchy, and choose XMLSpy as the destination. The Preview section at the bottom of the Export dialog shows a view into the table contents. DatabaseSpy Export Dialog When we click the Export button, DatabaseSpy formats the relational data with XML element names derived from the column names of the table and sends the resulting output directly to XMLSpy. The screenshot below shows a portion of the file in XMLSpy. The Message window at the bottom verifies the file is well formed. XMLSpy Editing window and Message window Note that DatabaseSpy supplied the root element <Import name = “cities”> and added comments to describe the datatypes of the database table columns. And, we did not have to construct a SQL statement with a cumbersome Concat( ) function. We began this post to address the simple requirement for a root element to complete the output of the Concat ( ) function we described earlier. When real-world projects require converting from relational databases to XML, the requirements are likely to be much more complex. Altova XMLSpy connects directly to all popular databases to work with XML technologies and relational data. XMLSpy lets you easily create an XML Schema from a database structure, or create a database schema from an XML Schema. XMLSpy also includes advanced editors and debuggers for XQuery and XPath for XML stored directly in databases, along with specialized support for XML features in Microsoft SQL Server, IBM DB2, and Oracle databases. As more industries adopt and evolve XML-based standards for information interchange, a common need is to convert data stored in legacy databases to XML. Altova MapForce connects to databases and allows you to map and transform relational data to be compatible with one or more XML Schemas. You can use your mapping to perform a one-time data conversion, you can save and re-open your mapping to perform another conversion later, or you can instruct MapForce to generate royalty-free source code from your mapping to include in your own project when repeated conversions are required. If you’d like to see for yourself how well Altova tools can generate well-formatted XML from relational databases, download a free trial of the Altova MissionKit.

Tags: , , , , ,

New XML Schema Editing Tools in XMLSpy


Let’s take a look at some of the new features for working with XML Schemas in the latest release of XMLSpy.

Sorting in Schema View

When you’re working in the graphical XML Schema editor, you can now sort some or all of the schema components alphabetically with a single click. This is a huge time saver when you need to organize large schemas that have evolved over time and for understanding new schemas that come across your desk. Simply click the A-Z button in the Schema Overview window and select the sort options you prefer. Your components are instantly sorted in the graphical view… xmlspy-sort-schema …and when you click over to Text View, the schema code itself is reordered and organized.

Schema Refactoring

Another new feature that’s invaluable when working with complex schemas or ones inherited from other developers is schema refactoring support. Taking advantage of XMLSpy’s powerful Find in Schemas functionality, you can quickly locate all the instances of a global element or type across a schema – and all imported and included schemas – and then rename that component in each place it occurs. This makes schema refactoring easy while ensuring the validity of all impacted schemas.

Intelligent Support for Changing Types

You may also find the need to change the type of an element or a base type, which previously meant reconfiguring all the facets associated with that component. This new feature gives you the option to preserve any facets or attributes that are compatible with the new type. Rename components to refactor schema  

Customizable XML Schema Documentation

Another option schema developers have requested quite often is the ability to customize the comprehensive XML Schema documentation generated by XMLSpy. This is now possible via integration with StyleVision. The StyleVision stylesheet design tool provides countless advanced options for customizing the documentation templates, from adding your company’s logo and branding to changing the appearance and organization of the documentation completely. StyleVision integration also gives you the option to generate documentation in PDF in addition to the HTML, Word, and RTF output options available for the fixed documentation in XMLSpy. Here’s an an example of XML Schema documentation we customized for the fictional Nanonull corporation: Embed image in XML XMLSpy 2011r2 also ships with several useful documentation templates that can be used as is or customized further in StyleVision:

  • XML Schema Structure Overview: documents the structure of global elements and complex types to the number of levels you specify
  • All XML Schema Type Derivations: displays all simple and complex types and their derived types in a tree format
  • XPath List: generates all possible XPath statements that are possible for a schema based on a user-specified number of levels
  • Statistics: lists the numbers of elements, types, attributes, groups, etc., for the overall schema, and for each associated schema file

To use a custom schema documentation template, you must have XMLSpy and StyleVision installed. If you are a MissionKit customer, you already have both tools and can take advantage of this feature and countless other options for working with XML Schema, from stylesheet and report design, to data mapping, and more. If you aren’t currently a StyleVision customer, grab a free trial from our Web site. The XBRL and WSDL documentation generated by XMLSpy may also be customized using StyleVision.   As always, we rely on your input and feature requests when planning each release – please let us know what you think of these new features and what you’d like to see in the next product version by leaving a comment here on the blog.

Tags: , , ,

Using Charts to Effectively Communicate Data


Altova first added support for charts and reporting the Altova MissionKit with the launch of Version 2011 last September. The v2011 reporting functionality includes options for line charts, 2D and 3D bar charts, 2D and 3D pie charts, round gauge and bar gauge charts. Here are a few examples: Charts created with the Altova MissionKit v2011

Advanced chart features in v2011r2

Version 2011 Release 2 of the Altova MissionKit, introduced on February 16, adds an exciting group of enhancements to the chart and reporting features in XMLSpy, StyleVision, and DatabaseSpy. The chart design options and user interface work the same way in all three applications, so MissionKit users can work intuitively and productively as they move from processing XML data in XMLSpy, to preparing charts for a business intelligence report with StyleVision, and even when they create graphical displays directly from SQL query results in DatabaseSpy. The wide range of new customizable charting features introduced in version 2011 release 2 includes:

  • Stacked Bar charts
  • Area charts
  • Stacked Area charts
  • Candlestick charts
  • Chart overlays
  • Background images and color gradients
  • Ability to change position of axis labels
  • And more!

Now you can create attractive and informative charts to represent a wide variety of data sets without exporting data to a dedicated charting application. Charts created using the Altova MissionKit are not limited to any specific presentation technology – for instance you can use StyleVision to include charts in HTML, Microsoft Word, RTF, or PDF documents, or you can save charts created in DatabaseSpy in a variety of image formats at the custom resolution you specify. In this post we will show some examples of the new charts and features available in all three MissionKit reporting and charting applications – XMLSpy, StyleVision, and DatabaseSpy.

Stacked bar charts

Stacked bar charts are a variation on bar chart presentation and are especially useful when multiple ranges of data need to be illustrated. Stacked bar charts are also useful to more clearly illustrate data in a smaller area. The image below shows a stacked bar chart to illustrate the performance of a sales team by region over two years Stacked bar chart Note that the combined height of each stack in the Stacked Bar Chart represents the total sales over the two-year period for each Territory, since the sales for Last Year are added above the Year To Date numbers. Stacked bar charts complement regular bar charts and 3-D bar charts to offer users the greatest flexibility in illustrating SQL query results. If the user prefers horizontal bars, a checkbox labeled Draw X and Y exchanged in the Change Appearance tab selects that orientation. Chart orientation option Horiztonal stacked bar chart This orientation option is also available for other 2-D bar charts, line charts, area charts, and candlestick charts.

Area charts

Area charts are similar to line charts, with shading applied to make a more graphically appealing display. The area chart below shows a record of temperature and humidity changes by hour over the course of one day. Creative application of color can emphasize the point! Area chart To successfully build an area chart, the analyst must consider the values in each data category. As the area chart is constructed, each category forms an opaque layer on top of the layers for data retrieved previously. In the case illustrated above, Temperature was always a larger number than Humidity, so a SQL query was constructed in DatabaseSpy to retrieve the Temperature value before Humidity to prevent Temperature from acting like a curtain to hide the Humidity data. However, if the data columns appear in a sequence with values in increasing order, the last layer would overlap and hide all the preceding layers. In that case, the chart tab heading titled Select Data lets the user add and delete columns from the results to re-sequence the data correctly. The Select Data column also lets the user edit the names assigned to each column on the X-axis label. Select Data dialog As alternative solution, the Transparency option in the Change Appearance tab lets the user adjust color levels to allow hidden layers to show through.

Transparency dialog

Stacked area charts

As implied by their name, Stacked Area charts layer the columns of a data set to illustrate the overall sum of a data series. Stacked Area charts also eliminate the potential overlapping data problem that can occur with regular area charts. The chart below shows a table of air passenger revenue miles traveled by month, with individual regions for domestic and international travel. Stacked area chart The Stacked Area chart creates a graphical representation of the total of Domestic and International miles, even though the total miles value was not part of the provided data. This is apparent at the top of the January entry, where the International region intersects the Y axis just below 600 (the original data showed 392 million Domestic miles and 181 million International miles, for a total of 573). A strategic data analyst will always consider the nature of the data to be reported when choosing any particular chart type. For instance in the weather example we used above, adding temperature and humidity values in a stacked bar chart would not be logical!

Candlestick charts

Candlestick charts were originally developed by a wealthy Japanese businessman who began trading at the local rice exchange around the year 1750. He kept records of the local market psychology, learning to boost his profits by carefully monitoring prices and not rushing into trades. Today, charts are used to represent financial data such as stock prices over a period of time. Every day the market is open, each stock has four relevant data points that can be rendered in a candlestick chart: the price at market opening, the price when the market closed, the high price during the day, and the low price during the day. Investors and financial analysts like to view these indicators to gauge the stock’s performance over a period of time. In the candlestick chart below, each solid bar represents the range between the opening and closing price and the thin vertical line through each bar shows the extent of the high and low prices for the day. Candlestick chart In this version of the chart, following common convention, the color of each bar signals whether the stock was up or down for the day. If the bar is green, the stock was up for the day– it opened at the price indicated by the bottom of the bar and closed at the price indicated by the top. If the stock was down for the day, the bar is red and the symbolism is reversed – the stock opened at the price indicated at the top of the bar and closed at the price shown by the bottom. Numerous options are available to set line and fill colors, the Y-axis range and values, and more. Because they were intended to be printed in black and white, the original candlestick charts used empty bars to indicate the price increased and solid bars to indicate price decreases. The Altova MissionKit offers this option: Candlestick chart in black and white Another candlestick chart variation omits the opening price and simply illustrates the range by a vertical line and the closing price by a horizontal line. This option is automatically supported when a data set only includes the high, low, and closing prices. Candlestick chart without opening price

Chart overlays

The Overlays feature lets you combine multiple charts in a single image. Each overlay chart has unique settings and can even be generated from a separate data file. The image below shows a candlestick chart of a stock’s daily prices with the daily sales volume in a bar chart overlay. Candlestick chart with bar chart overlay

Support for background images & color gradients

The ability to specify background color gradients and background images gives you even more flexibility for creating customized, eye-catching charts. Overlaying one chart on another lets you visualize multiple data sets with different Y-axes and types. Area chart with a background image The Change Appearance dialog lets users select a background image, as in the Winter Games chart above, or apply a background color gradient, as in the Summer 2010 chart below. Change Appearance dialog Bar chart with a line chart overlay and background color gradient If you’d like to see for yourself how easy it is to use Altova tools to create attractive charts from XML and database data, download a free trial of the Altova MissionKit.

Tags: , , , , , , ,

What Do Industry Authors Have to Say About Altova?


Authors of various industry reference books ranging from SOA and Web services to XML continue to use and recommended Altova tools. The latest update to the Cold Fusion book series – “ColdFusion 9 Developer Tutorial” is an update to John Farrar’s “ColdFusion 8 Developer Tutorial”. In this latest update, Farrar uses the Altova MissionKit, our suite of XML, database, and UML tools to do all his XML work for the book. According to Farrar, “I have a suite of tools from Altova and find they do what I want. I can create XPath, XML Schemas, and more from their tools and don’t ever feel the need to look for a new tool.” ColdFusion9_Farrar Farrar, a ColdFusion expert, teaches the basics of ColdFusion programming, application architecture, and object reuse. He then shows off a range of topics including AJAX library integration, RESTful Web Services, PDF creation and manipulation, and dynamically generated presentation files. So whether you need an overview of XML technologies, the latest information on working with ColdFusion, or want to delve into Web services, you’ll want to check out the Altova Reference Books page on our Web site.

Tags: , , , , , , , , , , , ,

MissionKit 2011 Now Available in Japanese


With the release of Version 2011 we are thrilled to bring you the Altova MissionKit in Japanese. Now all the Altova tools available in the MissionKit have been fully translated into Japanese. Like the English and German versions of the tool suite, the fully translated Japanese language version provides users with powerful functionality for XML and Web development, data mapping and integration, rendering and publishing of XML, XBRL, and database data, UML modeling, and more. All the tools available in the new Japanese language version of The MissionKit are available at the same cost as the English versions, and current Version 2011 users can now unlock any language version using their existing key code. missionkit_jp If you haven’t checked out our latest release – Version 2011, download a free, 30-day trial today! The Japanese language version of all the MissionKit tools can be purchased from the Altova Online Shop or through your preferred reseller.

Tags: , , , , , , , , , , ,

XMLSpy Version 2011


XMLSpy XML editor We recently blogged about the release of the Version 2011 MissionKit and all the new features it included. Let’s take a closer look now at XMLSpy 2011 and what’s new in the XML editor:

  • Creation of pie, line, bar, and gauge charts from XML
  • Validation of SOAP messages against the SOAP spec & corresponding WSDL file
  • Schema flattener
  • Schema subset picker

Creation of charts & graphs from XML data

Like several other tools in the MissionKit 2011, XMLSpy now supports chart and graph generation. Chart creation is unique to XMLSpy and brings a whole new dimension to working with data in an XML editor. Now you can visualize and analyze numerical XML data – and XBRL data – directly inside XMLSpy with just a few clicks. No more exporting to Excel! The following types of charts are available:

  • 2D & 3D pie charts
  • 2D & 3D bar charts
  • Category line graphs
  • Value line graphs
  • Gauge charts

Below are two example charts that show just some of the possibilities for representing XML data in attractive, meaningful way – with just a few clicks: Pie chart showing XML data Bar graph showing XML data To create a new chart, simply highlight a range of data in Text View or Grid View, right click, and select New Chart. You can also specify the data to chart via XPath expression. Once you’ve used the Select Columns dialog to specify which data you want displayed, either via XPath or by highlighting it in the window, and how you want to display it…. Select columns for XML chart …you can choose the type if chart you require, and the appearance of the chart, from colors to fonts and more. You can create up to nine different charts on the available tabs in the Output Window, and you can save each as a custom-sized image file with just a click or print it directly from XMLSpy. You can also copy it to your clipboard to paste it in another application, such as Microsoft® Word or PowerPoint. One of the coolest aspects of these charts is that they’re totally dynamic. Once you make a change to the underlying XML data, the chart can be instantly regenerated, either automatically by hitting Reload, depending on your preference. What’s more, you can copy the XSLT or XQuery code to create the chart to the clipboard for use in your own stylesheet or application. The XSLT/XQuery code can be executed in XMLSpy and via AltovaXML 2011 Enterprise Edition, Altova’s XML standards processor. Chart support extends to the XMLSpy XSLT profiler and XQuery profiler, as well. After profiling an XSLT or XQuery execution, one click creates a chart to help you immediately visualize the results based on one or more criteria. This makes it easy to immediately interpret the results of a profiling an XSLT or XQuery execution or to communicate those results with other team members in a powerful, visual way.

Validation of SOAP messages

Requested often by our customers, this new feature will be very useful for Web services developers, as it allows them to ensure their SOAP messages are valid against the SOAP specification as well as any XML Schemas referenced in the corresponding WSDL definition.

Schema flattener

The new schema flattener in XMLSpy 2011 allows schema designers to write all the dependent files that are included in a schema into one file. This way, you can use the flattened XML Schema without having to rely on a network connection to access the required files. This could be especially important when shipping a schema within an application. Schema Flattener

Schema subset picker

Many schemas are quite large and contain more types and elements than are required for a particular application or information exchange. Furthermore, the larger a schema is, the longer loading and validation may take. The ability to create a new schema from a subset means a developer can select some global elements or types from a large schema to create a smaller one, using just the elements required. This feature is especially useful for working with NIEM information exchanges, but has also been requested by other users for a variety of applications. Creating a schema from a subset   Read about all the features added to the Altova MissionKit tool suite or download the latest version now. If you’re a current customer with an active Support & Maintenance Package, you can update to v2011 for free.

Tags: , , , , ,

Using Altova Tools to Work with XML Data in the SQL Azure Cloud


In an earlier post we discussed connecting to Microsoft SQL Azure databases with Altova DatabaseSpy and demonstrated database schema comparison and content comparison between a local database and the same database migrated to SQL Azure. In this post we will use a different method to migrate an existing table to SQL Azure and show you some tricks you can do with XML in the cloud. We started by creating a new database schema in SQL Azure. Then we created a DatabaseSpy project with a connection to a local copy of SQL Server Express running the AdventureWorks sample database, and a second connection to our new SQL Azure schema. DatabaseSpy Project helper window The AdventureWorks database contains a table called JobCandidate with some XML data we will use for a model for our SQL Azure XML contents. AdventureWorks JobCandidate table in the DatabaseSpy Online Browser window We can generate a CREATE statement for the existing table to use as a basis for the SQL Azure version. CREATE statement for the existing JobCandidate table We need to modify this statement to execute in our SQL Azure database. In addition to changing the database and schema names, we will remove the foreign key constraint to the Employee table, since our new database doesn’t contain a table with that name. Also, SQL Azure does not support the CONTENT keyword, so we will remove that as well. Modifying the CREATE statement in a DatabaseSpy SQL Editor window After making sure the Properties window for the revised CREATE statement points to the SQL Azure database, we can execute the statement. When we refresh the database and expand our view in the Online Browser helper window, we can see the new empty table. New table viewed in the DatabaseSpy Online Browser window A data comparison between the existing table and the new one will allow us to create a script to migrate data into our new table in the SQL Azure cloud. This is similar to the data comparison we wrote about in our previous post on SQL Azure, except instead of merging data directly, we will save the merge script. DatabaseSpy Merge Data dialog Our first attempt to run the merger script failed, throwing an error message that SQL Azure cannot insert values into the new table when IDENTITY_INSERT is set to OFF. We can add a line to the merge script to SET INDENTITY_INSERT ON and re-execute: Merge data script viewed in a DatabaseSpy SQL Editor window Next, we can run a SELECT query to view the data that was successfully uploaded. Select query and Results viewed in DatabaseSpy The DatabaseSpy Data Inspector window lets us more easily examine the contents of a wide column, and is ideal to use for XML documents stored in the Resume column of the new JobCandidate table. DatabaseSpy Data Inspector window Editing XML Data with XMLSpy If you need to revise, edit, update, or validate XML data in a SQL Azure database, Altova XMLSpy provides more robust XML editing features than DatabaseSpy. We can connect to our SQL Azure database from XMLSpy and run a SELECT query from the XMLSpy Database Query window. XMLSpy lets us open any XML row for direct editing, with access to advanced XML editing functionality. XMLSpy Database Query view of XML data in the SQL Azure cloud Of course all the familiar features of the XMLSpy text view and grid view are available. XMLSpy Text view of XML data in the SQL Azure cloud XMLSpy Grid view of XML data inthe SQL Azure cloud After your edits are complete, the XMLSpy File / Save menu option saves the revised XML document to the same row of the JobCandidate table in the SQL Azure database in the cloud. XMLSpy File / Save menu option saves XML data to the SQL Azure cloud Parsing XML Data with XQuery You can also apply the XMLSpy XQuery editor, with its built-in knowledge of XQuery syntax and context-sensitive entry helpers to build XQuery statements that parse the XML data in your SQL Azure database. The XQuery statement below extracts and returns the home addresses from the XML resumes where JobCandidateID is less than 7. XMLSpy XQuery editor for XML data in SQL Azure The XQuery statement can be executed in the Database Query window, with results immediately available to work with in XMLSpy. XQuery statement and results inthe XMLSpy Database Query window Of course the XQuery result can also be edited in Text view or in Grid view. XQuery result in XMLSpy Text view XQuery result in XMLSpy Grid view And you can save the query result either from the Database Query window or from the XML Editor view. Find out for yourself how productive you can be by using Altova tools to work with XML data in the SQL Azure cloud ̶ download a free 30-day trial of the Altova MissionKit for Software Architects, an integrated suite that includes XMLSpy, DatabaseSpy, and additional XML, database, and UML tools.

Tags: , , , , , , ,

Altova Adds to NIEM Support in v2010r3


Clashes between government agencies are the stuff of legends and entire TV shows: the FBI and the CIA, the local sheriff’s department and the big city CSI, Homeland Security and the Pentagon, Jack Bauer and CTU. Perhaps this has to do with the territorial nature of some of these entities, but perhaps, just perhaps, technology has a part to play in these conflicts. Incompatible architectures and data formats, legacy systems, and other technology road blocks within the United States government sometimes made it difficult for agencies to efficiently share pertinent information such as arrest reports, amber alerts, immigration details, and more.
The National Information Exchange Model (NIEM) is an XML-based standard for data exchange between United States government agencies and their information partners. It aims to provide a means for these entities to improve decision making, achieve greater operating efficiency, mitigate risk, improve public safety, and increase ROI through intergovernmental information exchange.
As a leader in XML tools and technologies, the Altova MissionKit already inherently supports NIEM on a variety of different levels. In v2010r3, we added two very specific features to XMLSpy that extend the XML editor with capabilities that are invaluable for NIEM development. For more on these features, see the recent NIEM blog post by our CEO and XML Aficionado, Alexander Falk.
Below is a brief overview of the support provided for NIEM across the MissionKit, from XML Schema editing in XMLSpy, to XMI import and UML diagramming in UModel, to data integration in MapForce, and beyond.

XMLSpy

XMLSpy® 2010 provides a graphical schema design interface which simplifies the development of XML Schemas, allowing XML code to be generated in real-time behind the scenes. Users can switch back and forth between this view and text view at any time during their design process.

The graphical schema editor provides built-in support for creating the IEPD Extension Schema, Constraint Schema, and Exchange Schema.

clip_image001

In addition to the broad XML Schema support described above, XMLSpy provides comprehensive support for all other prevalent XML technologies, including:

UML Diagrams

The first stage in any development lifecycle is a thorough business requirements review. UML is particularly well-suited to modeling software projects because it provides a standardized approach to the design process. Scenario-based planning is recommended for NIEM, and though there is no UML diagram requirement, class diagrams, use-case diagrams, and sequence diagrams all provide value to IEPD documentation. In addition, NIEM provides a free tool for mapping data requirements based on an uploaded XMI representation of a UML model
UModel® 2010 is an advanced UML tool that supports all fourteen UML 2.3 diagrams, BPMN, SysML, and more. It works seamlessly with XML technologies like XMI, letting users easily import and export XMI based on their UML diagrams. UModel can even render XML Schemas in a format similar to UML diagrams and integrates seamlessly with all other MissionKit tools including XMLSpy, MapForce, and SchemaAgent. UModel also supports code generation, round-trip engineering, and reverse engineering, as well as integration with the popular Visual Studio and Eclipse IDEs.
UModel can be used two different phases of NIEM development:

  • Scenario Planning – to model and document with use case diagrams, sequence diagrams, and business process modeling
  • Map & Model – to develop class diagrams to create the exchange content model
clip_image002

XML Schema Management

Because of the potential to have many different schemas in each IEPD, XML Schema management can become an arduous process. SchemaAgent® 2010 is a unique file management tool that lets users visually manage their XML Schema, XML, XSLT, and WSDL assets in a project-based environment so that changes and structure of the file configuration can be easily recognized. SchemaAgent users can even view mapping files created using MapForce.

clip_image003

Data Mapping & Exchange

Data mapping is a crucial step at several levels in the NIEM process. Mapping helps users generate a subset schema to determine what elements can be used from the core and domain models, it is also used to perform a transform from one NIEM vocabulary to another. Some NIEM implementations also employ Web services to provide a mechanism for the exchange.
MapForce® 2010 is an any-to-any data mapping and integration tool that supports all of the potential data formats used in the NIEM lifecycle including XML, databases, flat files, Excel 2007+, and Web services. MapForce maps data based on its underlying structure (i.e. XML Schema), enabling mapping designs to be reused for recurrent transformations. MapForce also supports mapping to and from multiple source or target components. In addition, MapForce can be used to generate documentation detailing mapping projects for non-technical stakeholders in a variety of formats such as HTML and Microsoft® Word.

clip_image004

To try out this functionality on your own, download a free 30-day trial of Altova MissionKit now!

Tags: , , , , , , ,

Altova MissionKit Integration with Visual Studio – On with the Show!


VisualStudioWebButton Altova products have long supported tight integration with Visual Studio, giving developers seamless access to the advanced functionality of XMLSpy, MapForce, StyleVision, and UModel directly in their preferred development environment.So naturally, the Altova team is excited to attend the launch of Visual Studio 2010 on April 12-14 at the Bellagio Hotel and Casino in Las Vegas, NV. We’ll be demonstrating all the latest features of the Altova MissionKit 2010 Release 2 and we’d love to meet you at booth 614 on the Exhibition floor.If you’re not headed to Las Vegas, you can read more about Altova Integration with Visual Studio at the IDE Integration page on the https://www.altova.com/. Or follow the links below for detailed information on Visual Studio integration for each tool in the MissionKit:XMLSpy 2010 Professional and Enterprise Editions support seamless integration with Visual Studio , Microsoft’s premier application development environment, where you’ll have access to all of XMLSpy’s tools and utilities for modeling, editing, transforming, and debugging XML technologies.MapForce 2010 Professional and Enterprise Editions support full Visual Studio integration, allowing you to harness the power of MapForce for data integration, Web services implementation, and C++, C#, and Java code generation directly inside Visual Studio.StyleVision 2010 Professional and Enterprise Editions are offered as Visual Studio plugins, meaning that you can easily design and implement stylesheets for simultaneous XML, XBRL, and database output to HTML, RTF, PDF, and Word 2007+ (OOXML) directly within Visual Studio.UModel 2010 Enterprise Edition now supports advanced integration with Visual Studio, including code/model synchronization, giving you access to all of its UML modeling capabilities from within Visual Studio.(Oh yes, we know Visual Studio is not the only tool for all the XML, data integration, and UML developers out there. The Altova MissionKit IDE integration modules also support Eclipse.)

Tags: , , , , , ,

A Developer’s Life is Never Simple


Earlier this month in our blog post on Comparing XML Schemas we showed a realistic – but simplified – example to illustrate a slick new feature of DiffDog 2010 to compare XML Schemas and update corresponding XML data files by generating XSL transformations.
A real-life XML developer’s project is rarely as small and straightforward as the example we used. In this post we will take a look at some typical complications developers face every day and how the Altova MissionKit cuts through complexity to enhance productivity.
Longer XML Schemas and Larger Mappings
Your XML Schemas are not likely to be as short and simple as the ones in our earlier post, and you will likely want to save your work while you are mapping your XML Schema migration. DiffDog lets you save your mapping in an XML Schema comparison file that you can reload later to continue your work, or to share with a colleague.
DiffDog Save XML Schema Comparison dialog
If you need to compare two XML Schemas on a regular basis, the XML Schema Comparison Document can be a valuable time saver.
Embedded XML Schema Assignment
The screen shot below shows the XML data file from our original example with one important difference. In this version the story element includes an embedded reference to the XML Schema on line 2.
XMLSpy XML Editor view
When we transform this file with the simple XSLT we created in DiffDog, the new XML data file will not include the updated XML Schema reference. In our earlier blog post embedded XML Schema references weren’t an issue because we used the XMLSpy Project / Properties menu option to assign default XML Schemas for each folder.
If there are many XML data files to transform, and they require embedded XML Schema references, we can take advantage of an additional feature of MapForce. We can export our mapping from DiffDog to MapForce, as we did in the earlier post, and then use the Component Settings dialog for the MapForce output component to include the XML Schema reference.
MapForce Component Settings dialog
Document Your Work
Developers can lose productivity trying to retrace history when a project needs additional work months or even years after an early iteration. The MapForce Generate Documentation feature can help us avoid this frustrating experience.
MapForce will document the mapping of each element in the XML Schemas in Microsoft Word, RTF, or HTML formats. Regardless which format we choose, the resulting document is an excellent stand-alone project artifact, or it can be further edited and included in a larger report.
MapForce Generate Documentation dialog
We can even combine the MapForce mapping documentation with full descriptions of each version of the XML Schema generated with the XMLSpy XML Schema Editor documentation feature.
Source/Version Control
Source/version control systems let teams of developers work closely together on the same project without a risk of overwriting each others’ changes. Because a version of the source code is saved at each stage of the design process, it is very easy to look at or revert to an earlier version when needed.
Altova has implemented the Microsoft Source Code Control Interface (MSSCCI) v1.1 – v1.3 in XMLSpy and tested support for many popular source control systems, so we can manage the files in our XML Schema evolution project across the development enterprise. Additionally, DiffDog can be integrated with source control systems as the default comparison tool. DiffDog can even generate differences report files in a variety of formats.
See for yourself how the tools in the Altova MissionKit can cut through the complexity of your own XML, Web Services, data integration, XML publishing, XBRL, and UML modeling development projects – download a free 30-day trial!

Tags: , , , , , , , ,

Comparing XML Schemas with DiffDog 2010


DiffDog 2010 includes a powerful new tool to compare XML Schemas that XML developers and others can use to update existing XML data files as XML Schemas evolve. This post takes a look at an example scenario for this feature.Before we drop into the new functionality, let’s take a quick look at two XML Schemas using the DiffDog File Compare feature. Of course, just like in previous versions, DiffDog 2010 users can compare XML Schemas as .xsd documents and display differences in a color-coded, XML-aware format.DiffDog file comparison view of XML Schemas This is a good way to identify and manage differences in XML Schemas, especially when you want to review revisions to industry-standard XML Schemas that evolve over time.What’s new in DiffDog 2010 is an additional XML Schema Differencing option that graphically displays two XML Schemas side by side, identifies identical elements automatically, and lets users map differences and generate XSL transformations to update XML data files.Here’s our first view when we open the same two XML Schemas shown in the file comparison above, using the new XML Schema Differencing feature.Initial DiffDog XML Schema Differencing view of XML Schemas The root elements of the two XML Schemas are automatically connected. We can click the Compare button in the toolbar to automatically connect identical elements in the two XML Schemas.DiffDog XML Differencing (Of course we could also select Compare XML Schemas from the right click context menu, or choose Start Comparison from the Diff and Merge menu, or press the F5 keyboard shortcut – DiffDog gives you many options to perform the same task, so you can work the way you like.)Next, we can map elements with different names in the two XML Schemas by manually connecting the pointer arrows between them. In this example most of the changes to the version of the XML Schema on the right simply give elements new names that will be more clear when the XML Schema and its data files are distributed through our enterprise.User-mapped XML Schemas in DiffDog XML Schema Differencing view When all the elements are mapped, we can generate an XSLT file to transform existing XML data files based on the XML Schema on the left to reflect revisions in the newer version on the right. This feature is designed to rescue XML developers from the tedious tasks of writing and debugging XSL transformations by hand.DiffDog Diff and Merge Menu Here is an example of an original XML data file based on the XML Schema on the left side, as viewed in Altova XMLSpy:XML data file viewed in XMLSpy The output file after applying the XSL transformation we created with DiffDog 2010 appears below. Note the substitution of the author element for writer, email for feedback, and so on.XSL output viewed in XMLSpy If there are many existing XML files that need to be transformed, the Project Management features of XMLSpy can help us automate the process. We can add external folders to an XMLSpy project.XMLSpy Project Helper Window Using the XMLSpy properties dialog for each project folder, we can assign default values to assign an XML Schema for validation, the XSL transformation, and the destination of the output.XMLSpy project folder properties dialog Now we can select the input folder in the XMLSpy Project helper window and transform all the files in it with the single-keystroke F10 shortcut.When we originally mapped the XML Schema elements in DiffDog, we left the publication element on the left side unconnected, since it had no corresponding element in the earlier version of the schema. That means when we transform XML input files using the XSLT, the resulting output will not contain the publication element. If publication is a required element, we can call on Altova MapForce for a quick solution.One of the options in DiffDog is to generate a MapForce mapping rather than XSLT. When we choose this option, MapForce launches with our DiffDog mapping already loaded as a new MapForce design, as shown below.MapForce New Design It’s easy to enhance the mapping by adding a constant as a default value for the publication element.MapForce enhanced design Now we can save an XSL file from MapForce that reuses all the element mappings we originally designed in DiffDog and adds the constant. When we apply the new XSL to transform our original XML data file, we get a result that includes the default value for the publication element.Final version of output viewed in XMLSpy This post started by describing the new XML Schema Comparison feature in DiffDog 2010. Fleshing out a simple – but typical – real-world example quickly highlighted additional tasks easily completed by taking advantage of tight integration with XMLSpy and MapForce.All three of these tools and more are available at substantial savings in the Altova MissionKit 2010, the integrated suite of XML, database, and UML tools designed to meet the diverse development and data management needs of today’s software architects and XML developers. Click here to download a free trial today!

Tags: , , , , , , , , ,

Report from Microsoft PDC


We always enjoy meeting developers who currently use Altova tools and others with projects our tools can help them complete. This year’s PDC in Los Angeles was no exception – great weather, great camaraderie, and a brand new Version 2010 of the Altova MissionKit to demonstrate and talk about. Below is our short YouTube video of PDC highlights. If you were there, see if you can spot yourself in the crowd. If you didn’t get to go this year, we’re sorry we missed you.  

  Wanted2 And don’t forget to check out Version 2010 of the Altova MissionKit online. Version 2010 is packed with over 70 new features that were requested by our current users. Our What’s New page describes highlights of the major new functionality in XMLSpy and the other Altova developer tools. If you are covered by a current SMP plan, your update to v2010 is free. If you need to purchase an upgrade, click here to visit the Upgrades page on our Web site. The Altova Upgrades page describes all the details and connects to our online Upgrade Wizard to get started right away. You may be eligible for a discount of up to 40%! Our trip to Microsoft PDC wraps up the Altova show season for 2009. We hope to see you in person at another event next year.

Tags: , , , , , , , ,

Altova at Microsoft PDC


clip_image001The Altova road trip continues as we head west to Microsoft PDC in Los Angeles from November 17-19 at the Los Angeles Convention Center. If you’ll be attending PDC, make sure to stop by and meet with the Altova team at booth 517. We’ll be demonstrating all the Most Wanted features of Version 2010, our latest software release that includes XBRL enhancements in XMLSpy, support for WSDL 2.0 in XMLSpy and MapForce, a new absolute positioning design paradigm in StyleVision, database schema conversion in DatabaseSpy, and much more. With the new emphasis on software modeling in Microsoft development tools, you’ll want to check out SysML and all the other new functionality in UModel 2010. We love L.A., but if you’re not going to PDC this year, be sure to check the Altova blog again later for updates about the event and more details about Version 2010.

Tags: , , , , , , , , , , ,

XMLSpy’s Most Wanted


Altova's Most Wanted When we announced that Altova Software Version 2010 included over 70 of the features most requested by our customers, we weren’t talking about little tweaks and enhancements, but major new functionality! To show you we mean business, I’ll outline some of the most requested features added to XMLSpy 2010 here.

WSDL 2.0 Support

In response to requests from from Web services developers, the graphical WSDL editor in XMLSpy 2010 now supports the latest version of the WSDL standard, WSDL 2.0. This adds to existing support for WSDL 1.1, giving you the choice of which version of the standard to work with. The WSDL editor automatically provides the correct editing environment for the version currently being utilized, and XMLSpy even provides one-step conversion capabilities for migration between WSDL 1.1 and 2.0. Since the XMLSpy WSDL editor uses a graphical interface (you can, of course, also work in Text View if you wish), you can easily visualize the structure of your WSDL document and edit it using drag-and-drop functionality and context-sensitive entry helpers, which offer the relevant choices based on the selected WSDL version.

WSDL 1.1/2.0 editor

Enhanced XBRL Functionality

Since we added support for XBRL validation and XBRL taxonomy editing in XMLSpy 2009, we’ve received excellent feedback from customers, including some feature requests that we were able to address in v2010. The new XBRL documentation generation capabilities of XMLSpy 2010 make it easy to generate comprehensive documentation – in RTF, MS Word, or HTML – for your XBRL taxonomies. Multiple options let you choose exactly what to include in the documentation, and the resulting output (snippet shown below) includes hyperlinked components for easy navigation. XBRL taxonomy documentation

Another option for documentation is to print the graphical representation of your taxonomy as it is shown in XMLSpy’s graphical XBRL view. XMLSpy 2010 also includes the new XBRL Taxonomy Wizard to give you a head start when creating a taxonomy. Simply enter the company name, ticker, or other identifier for your XBRL taxonomy, and then select the base taxonomy to extend (if any).

XBRL Taxonomy Wizard

XMLSpy creates the required taxonomy files and prompts you to select the entry points of the base taxonomy. Once you click finish, XMLSpy 2010 displays the newly created XBRL taxonomy files in the graphical XBRL Taxonomy Editor, where you can continue editing and refining the taxonomy in a visual manner. The new Find in XBRL and XBRL Sort options in XMLSpy 2010 meet customers’ requests for quick, easy ways to find data in and

navigate through large, complex XBRL taxonomies.

XBRL Taxonomy Editor

JSON Editor

We’ve recently heard from a lot of developers working on Web 2.0 and Web services apps in XMLSpy who also use JSON – so we decided to add a JSON editor in XMLSpy 2010. You can compose JSON strings in Text View or Grid View, and even convert between XML and JSON. In Text View, the JSON editor provides syntax coloring, line numbering, source folding, bookmarking, and more, making it easy to comprehend and navigate your JSON code, and find and edit strings. Intelligent JSON editing populates the Elements entry helper window with a dynamically built list of the elements present in your JSON file, which you can insert with a double-click.

JSON editor text view

Intelligent JSON editing is also available in Grid View, which provides graphical representation that shows the structure / outline of a JSON document through a set of nested containers. These can be easily expanded and collapsed to get a clear picture of the document’s tree structure, and drag-and-drop editing is supported.

JSON Editor Grid / Outline View A final must-have feature for working with JSON is the JSON <=> XML converter in XMLSpy 2010. One click lets you, for example,  convert an XML file to JSON for transport with JavaScript, or convert data received in JSON format to valid XML. No more JSON vs. XML arguments – XMLSpy gives you the best of both worlds.

Redesigned Scripting Environment & Forms Editor

XMLSpy includes an integrated scripting environment and forms editor that has been redesigned for this latest release. Scripts can be written in JScript or VBScript to access and interact with the XMLSpy API, allowing you to modify and add functionality to your installation of XMLSpy 2010. Improvements and optimizations in Version 2010 include:

  • Access to most of the .NET framework
  • New form editor controls
  • Testing & debugging of macros directly in the scripting editor
  • Execution of macros directly through XMLSpy menus
  • Improved entry-helpers & auto-completion in the scripting editor

Read more about the “most wanted” features in XMLSpy and the rest of the Altova MissionKit. Please be sure to let us know your most wanted features, either by commenting here on the blog or entering a feature request.

Tags: , , , , , ,

NYC & Company Case Study


Overview NYC & Company is the official marketing, tourism and partnership organization for the five boroughs of New York City. Its mission is to maximize travel and tourism opportunities, build economic prosperity, and spread the dynamic image of New York City around the world. In 2008-2009, the company initiated a major rebranding, redefining their Web presence and launching an interactive multi-media center in Midtown Manhattan. At the center of this transformation, NYC & Company used development tools from the Altova MissionKit – UModel, DiffDog, DatabaseSpy, and XMLSpy. The NYC & Company Web site and Information Center was created together with online powerhouses as Google and Travelocity, reservation sites like Open Table, content providers Time Out, Greenopia.com, the New York City Department of Cultural Affairs, and more. The Challenge As the single organization responsible for meeting the marketing and tourism needs of the city of New York, NYC & Company has been tasked with meeting Mayor Bloomberg’s January 2006 State of the City goal of luring 50 million visitors by the year 2015 – up from an estimated 43 million in 2006. A large part of the effort behind this push would be manifested in a general Web site rebrand/redesign coupled with the creation of an interactive visitor center. NYC & Company chose to use existing tools and technologies as much as possible, leveraging their ColdFusion Web site architecture, the Eclipse software development platform, a SQL Server 2005 backend, and the Altova MissionKit. A new content management system was also implemented to manage the large amounts of data and associated workflow. The Solution The NYC & Company Web site redesign included a migration from nycvisit.com, which followed a typical convention and visitor bureau site structure, to the much more animated and multi-faceted nycgo.com, a design that promotes the dynamic nature of the resources available and of the city itself. clip_image001 nycvisit.com on 26 February 2008 clip_image003 nycgo.com on 22 May 2009 UML Modeling The new design components were drawn out as a UML class diagram, expanding on the data model that was created for the live Web site. NYC & Company used Altova UModel to map out the physical structure of nycgo.com, importing their XML Schema definition to ensure adherence to formatting rules. The class diagram was used to represent the new Web site structure at a high level, and to model the objects that needed to be built into NYC & Company’s content management system (CMS). UML design in UModel also enabled the company to generate documentation so that the developers could share the UI design with those not familiar with the intricacies of UML. clip_image004 UModel UML Class Diagram of the nycgo Web site NYC & Company then worked with third party design vendor, HUGE, Inc., to further analyze the UML wire frames and predict user interaction scenarios for the nycgo Web site. Dynamic code was then delivered in JSP, implemented on JRun then subsequently converted to ColdFusion. Code Differencing NYC & Company chose to migrate their JSP templates to ColdFusion 8 for its rapid application development capabilities, rich feature set, and intrinsic simplicity. DiffDog, Altova’s diff/merge tool, was an integral part of the development process, helping the development team to ensure that the ColdFusion code was in line with the original JSP. NYC & Company could easily recognize and reconcile any crucial differences using DiffDog’s straightforward text comparison interface. diffdog2 JSP/CFM code differencing in DiffDog Database Migration As part of their rebranding effort, NYC & Company successfully migrated their data from SQL Server 2000 to SQL Server 2005. NYC & Company used Altova DatabaseSpy to connect to the database, structure queries, and for database analysis. They also use the integrated SQL Editor to test their more complex SQL queries. This enabled them to do their database management and testing in-house, with non-technical and even non-DBA team members assembling complex SQL scripts with features such as auto-completion, syntax color coding, automatic formatting, and refactoring. Building Out the Content Management System NYC & Company used a third party CMS to manage workflow and collaboration for newly designed the Web site. The CMS was also modified to output XML feeds. Additionally, content sourced from NYC & Company’s partners was validated against an XML Schema and then imported into the CMS. Every night, a scheduled task is initiated that delivers the formatted XML feeds to the interactive data center. XMLSpy, Altova’s XML editor, provides NYC & Company with all of its XML editing needs – from validating and saving content, to managing and manipulating it as part of an integrated workflow. Real-time XML Feeds The XML feeds that are available on the nycgo Web site, and the interactive wall kiosks and tables at the Information Center are taken from data submitted by NYC & Company’s numerous content partners and provide real-time information about attractions and events all around the city. Once accessed, the information can be transferred to any mobile device via SMS. The walls display touch-screen FAQ stations that inform visitors about top New York City attractions and provide other useful information like how to tip a doorman, places to exchange currency, etc. in English and nine other languages. Users can also buy MetroCards and tickets to exhibits and other popular events. ONIC-launch-007 The same real-time data is also fed to interactive tables, where visitors place a “puck” on a Google map of the city to select their area of interest. They then click on a category (e.g., dining, entertainment, etc.) to get more information. clip_image010 The Results NYC & Company offers the latest in travel and tourism to New York City’s visitors, which number well over 40 million in any given year and offers a wealth of new experiences and up-to-date information to adventurous locals. The innovative new Web site design and interactive exploration center pulls together the latest in hardware, software, and data management technologies to showcase every aspect of this multi-faceted city to tourists from all walks of life and with all sorts of interests. NYC & Company was able to leverage the Altova MissionKit to manage large amounts of disparate data from a variety of different sources -from the preliminary UML modeling, to code differencing, database management, and XML editing. Find out how the Altova MissionKit can help with the end-to-end management of all of your data assets. Download a fully functional free trial of the Altova MissionKit today!

Tags: , , , , , ,

New Online Training – XMLSpy XML Editor


The Altova Online Training team is very excited to have just launched the much-anticipated first module in the XMLSpy training course! XMLSpy Module 1* provides an introduction to XML and the XMLSpy XML editor:   xmlspy-training   In this beginner-level module, students start with an overview of XML,  including the anatomy of XML documents and schemas. After a brief tour of the XMLSpy user interface, you’ll create an XML Schema and walk through the steps of defining a namespace, creating a content model, adding elements, configuring schema views, and generating sample XML files and schema documentation. Then it’s time to create an XML document based on the schema. By the end of this module, you will be able to enter data in XMLSpy’s grid view and text view, perform well-formedness and validity checks, add new elements, and modify your schema while working on our XML document. Detailed tutorials walk you step-by-step through each task, and you can test what you’ve learned using the interactive quizzes for each chapter. Check out the free XMLSpy training module* now, or visit the Altova Online Training page for a complete list of available training topics, including MapForce, StyleVision, XBRL, and more. All Altova Online Training courses are available on-demand and free-of-charge.   *See Altova Online Training System Requirements for supported browsers, etc.

Tags: , , ,

Wrycan / NAVSEA Case Study



Overview

The Portsmouth Naval Shipyard in Kittery, Maine, is a division of Naval Sea Systems Command (NAVSEA), the largest of the United States Navy’s five systems commands. They approached Wrycan, an Altova partner focused on content-centric XML expertise, for help converting some of their legacy format technical manuals to XML based on the Navy ETM XML DTD and recreating them as PDFs. The shipyard had been given a mandate to start utilizing XML as their primary data and storage format and needed a low cost and reliable publishing solution that could be easily maintained by their in-house workforce. Wrycan had some experience working with the Altova MissionKit for XML development, as well as a broad expertise in XML technologies including XML, XSL:FO, and DTD. They chose to use XMLSpy, StyleVision, and Authentic as the development tools for this implementation because of their intuitiveness, ease-of-use, and low price tag.

The Challenge

The Portsmouth Naval Shipyard needed to convert about 10,000 pages of content from a legacy format into XML that was conformant to their DTD. This included an automated conversion, manual review and cleanup, and a command line tool to publish the XML back into its original PDF format. As with any large publishing and conversion operation, the project required heavy QA review post-conversion, much of which could be done by non-technical shipyard employees if they had a mechanism to help them interpret and access the XML markup. In addition, because of the relative complexity of the documentation format, which included complicated page layout details such as a variable number of columns per page and different margin widths, callouts interspersed with sections and enumerated lists, as well as many large schematic models, some of which were on foldout pages, the XSL:FO coding promised to present a formidable challenge.

The Solution

Wrycan performed the bulk of the content conversion in-house using custom scripts and some manual processes, along with some technical QA. After the content was converted, Wrycan used StyleVision’s drag and drop design interface to create Authentic e-Forms for editing using the Navy ETM XML DTD as the structural component. Advanced stylesheet functions such as conditional templates and auto-calculations were inserted to facilitate QA and editing workflows. navsea_design After the content conversion, Wrycan implemented a command line processing tool that includes multiple steps such as:

  • Volume assembly from chunks of XML files
    For greater flexibility and usability, the Navy technical manuals were divided up into sections including Front Matter, Chapters, Back Matter, and image files. This enabled Wrycan to make certain parts of these files available for reuse. Components that appeared identically in more than one place within the manuals could be segmented so that changes made in one place would iterate throughout the documentation.
  • XML to XSL-FO conversion
    Wrycan used XMLSpy, Altova’s full-featured XML editor , to hand-code the advanced XSL:FO that was needed for the manuals. The complexity of the XML and PDF output can be seen in the following examples: Volume source, Front Matter source, Chapter source, and Final document (3.8 MB PDF).
  • Custom page formatting
    This project required various page sizes within one document, such as a portrait page followed by a foldout 11″ x 17″ landscape page. There are Naval documentation requirements specifying that different page formats have different printing requirements. For example, foldout pages are printed on one side only while other pages are double-sided.
  • Post processing steps
    There were also page numbering requirements, such as every chapter must start on an odd numbered page. If this causes a page to be blank, a message indicating that the page was intentionally left blank is placed on the page. These requirements are automatically satisfied by Wrycan’s processing tool.
  • PDF creation
    Wrycan integrated RenderX’s XEP software into the processing pipeline to convert the XSL:FO output, including all images and common content, into one PDF file.

The editing of the content is done with Authentic via Stylevision, which was recently upgraded to the most recent release for more advanced table support and authoring options. Below is a sample screenshot of one of the Authentic e-Forms for WYSIWYG XML editing that was generated for NAVSEA based on the StyleVision stylesheet design. navsea_doc

The Results

The Portsmouth Naval Shipyard now has an XML publishing solution with native XML editing capabilities. They can reproduce their technical manuals in PDF using XML as the content source. They are now ready to move onto the next step, which is implementing a full scale content management system with workflow and custom publishing capabilities. Find out how Altova tools can help with your documentation and publishing challenges. Download a fully functional free trial of the Altova MissionKit today!

Tags: , , , , , , ,

Internationalization with the Altova MissionKit


The following post is written by Peter Reynolds, CEO and translation management consultant at TM-Global and Executive Director of Kilgray Translation Technologies. An Irish national based in Warsaw, he holds a BSc and an MBA degree from Open University and is a localization and translation industry veteran. Peter previously worked at Idiom Technologies Inc. — now SDL PLC. As director of the LSP Partner Program at Idiom, Peter was responsible for making its global LSP partners program a successful and innovative venture. Before Idiom, he worked on language technology development for several global localization companies: Lionbridge, Bowne Global Solutions and Berlitz GlobalNET. He managed the Dublin development team responsible for BerlitzIT, Elcano, Freeway 2.0 technology solutions, and internal project and vendor management tools. Peter has been actively involved in the development and promotion of standards (notably XLIFF) for more than ten years, mostly at OASIS. Until 2008 when XLIFF was published, he was secretary of the XLIFF Technical Committee at OASIS and chaired the Translation Web Services TC. He is currently involved in OASIS, TILP as well as being the Irish expert to ISO SC2 and SC4 and training auditors for the EN 15038 standard.

Introduction

Every developer wants his or her applications to be used and hopes they will be very popular. A web application developed in rural Maine USA could easily be used by someone living in the next township or in Malaysia, New Zealand, Germany or Poland. Even if the application is not translated (localized), there are some important differences between how data is represented from one locale to another. The W3C definition of internationalization is “the design and development of a product that is enabled for target audiences that vary in culture, region, or language”. This does not mean that the product has to be translated into the language of the target audience but that it is designed in such a way that the target audience can use the application and understands the way data is presented. The reason for internationalization is to ensure the widest possible audience for your application and to make its translation easier and less costly. This article will introduce you to internationalization and demonstrate how applications can be internationalized using the Altova MissionKit, an integrated suite of XML, database, and UML tools including XMLSpy, StyleVision, MapForce, and others. If you are using tools such as XMLSpy and StyleVision it is very likely that you are already creating internationalized XML applications. The strategy which I suggest is that you try and figure out what target audience your applications are intended for beforehand and implement internationalization accordingly. In this article I will first discuss a strategy for internationalizing XML. I will then introduce the Internationalization Tag Set and examine issues relating to XML internationalization.

Strategy for Internationalizing XML

The first step in planning internationalization is to make an informed decision as to the level of internationalization you require. There may be people in your organization who can help you make this decision, and it would be particularly useful to obtain input from people who live in different countries. The three-level approach presented below should help you decide on the level of internationalization you are going to implement. However, you should remember that you may encounter some problems if your documents or applications are not internationalized, but you will certainly not have the same problems if to ensure that they are fully internationalized. The three levels of internationalizations are:

  • Level 1 – Your applications are likely to have a relatively small audience, which could grow, but the applications are unlikely to be translated or used internationally. In that case you should just follow the suggestions in this article and ensure that you use the functionality in Altova MissionKit to support internationalization.
  • Level 2 – Your applications will have a wide audience and could be translated and used internationally. As well as using the Altova MissionKit functionality you should also use the Internationalization Tag Set. This is a schema released by the W3C for the purpose of internationalization.
  • Level 3 – Your applications are most likely to be used internationally and translated into a number of different languages. You should consider how to improve the localization process by separating content from code and ensuring the translators can see the document or application as the end user would see it. This is beyond the scope of this article but you will find some relevant information on the subject in the references below.

The software tools in the Altova MissionKit have a lot of functionality which supports internationalization. If you are using these tools you have a very strong basis for creating internationalized XML documents. Unicode is the default encoding for applications created in the XMLSpy XML editor, and I would strongly recommend using this character set.

Internationalization Tag Set

The Internationalization Tag Set (ITS) is recommended by W3C and designed to create XML which is internationalized and can easily be localized. If you are working with XML documents which might be localized, I would recommend using ITS. With this technology you are able to specify which text requires translation, provide instructions for translators and specify the direction of the text. The seven data categories included in the ITS are:

  • Translate: Defines which parts of a document are translatable.
  • Localization Note: Provides notes and helpful information for translators.
  • Terminology: Identifies terms in the documents.
  • Directionality: Indicates the direction which the document or part of the document is written and should be read.
  • Ruby: Indicates which parts of the document should be displayed as ruby text. (Ruby is a short run of text alongside a base text, typically used in South-East Asian language documents to indicate pronunciation or to provide a brief annotation).
  • Language Information: Identifies language used for the different parts of the document.
  • Elements Within Text: Indicates how elements should be treated with regard to linguistic segmentation.

W3C has published a best practices guide for internationalizing XML documents which details how to use ITS. It can be found on their web site at: http://www.w3.org/TR/2007/WD-xml-i18n-bp-20070427/ The specification can be found in this section: http://www.w3.org/TR/2007/REC-its-20070403/ I would strongly recommend you read these documents before proceeding with internationalization.

Internationalization Issues

The following table describes some of the internationalization issues you may come across. This will be followed by a more detailed explanation of these issues and suggestions for how they can be resolved using the Altova MissionKit. .

ISSUE DESCRIPTION
Encoding Characters need to be supported by the code page being used. Unicode is an encoding which supports characters from all common language.
Date & TimeHow dates and time are represented varies between countries.
NumbersHow decimal points and thousands are represented varies between different countries.
CurrencyAs well as difference with how the number is represented in some countries the currency symbol or word is written after the number while in most it is written before.
Salutation & Names There are many differences in salutations between countries, and in some countries, such as Hungary, a person’s name is written with the family name first. No middle name is used in Japanese.
AddressThere are a number of differences relating to address, such as the house number appearing before the street name in some countries and after in others. Also, some countries use a ZIP code vs. a postal code.
RTLText is many languages is read from left to right, but in some, such as Hebrew and Arabic, the text is read from right to left (bi-directional).
Sorting & Collation There are differences in how alphabets are sorted. Some Scandinavian languages have an ‘aa’ character which is usually, but not always, sorted at the end of the alphabet.
Exclamation & Question MarksIn English questions and exclamation marks are always at the end of the sentence, while in Spanish there is a question mark at the beginning and end of a sentence.

.

Encoding

All electronic text uses a character coding system where the character is represented by a number. Before the widespread use of Unicode this was one of the most significant internationalization issues. When an application tries to show a character that is not represented in a code page it will appear as garbage text. There were not only problems between different languages but also with characters appearing incorrectly on computers running different operating system. Unicode has solved most of these problems by creating a single code page regardless of platform, program or language. XML uses Unicode as its default code page. Any XML documents you create in XMLSpy will by default have the declaration encoding="UTF-8” If the file has not been created in XMLSpy, you need to ensure that the file is saved as UTF-8. UTF is an acronym for Unicode transformation format, and UTF-8 is a flavor of Unicode that uses 1, 2 or 4 bytes to store characters. It is the most commonly used flavor and is very widely used for XML and the Web. The other versions of Unicode which XMLSpy supports are:

  • UTF- 7. This is 7 bit version of Unicode. It should only be used in the context of 7 bit transports, such as email.
  • ISO 1064 UCS – 2 and UTF – 16. UCS is an acronym for Universal Character Set and UCS-2 uses two bytes for each character. UTF-16 is an extension of UCS-2 which uses 2 or 4 bytes to represent a character. UTF-16 is often used by Windows and Java. You should use UTF – 16 rather than UCS – 2 for new documents.
  • ISO 1064 UCS- 4. Uses 4 bytes for each character and is the same as UTF-32. UTF-32 is often used by Unix.

There may be reasons for using default encoding other than UTF-8. To set the default encoding in XMLSpy go to Tools | Options and select the encoding tab.  XMLSpy encoding options If you want to change the encoding for an individual XML document, open the document in XMLSpy and select File| Encoding. XML encoding options

Language

The XML namespace defines xml:lang to identify the language of an XML document. The value for xml:lang must be an ISO language code (ISO 639- 2). If you have an XML document which is written in one language but has a segment in another language you can use xml:lang at the root element to identify the main language of the document and use it at the element where the text in another language is used to identify that language.

Dates

In different countries dates and time are represented in very different ways. Let’s take as an example the date 10/09/08:

In most European countries this means the 10th of September 2008.
In the United States this means the 9th of October 2008.
In Japan this means 8th of October 2009.

The way to deal with this is to use ISO 8601 for specifying date and time within your application. This is a standard way for representing date and time in the format YYYY-MM-DDTHH:MM:SS[±HH:MM] where

YYYY- represents year
MM- represents month
DD – represents day
T signifies that Time follows this
HH- represents hours
MM- represents minutes
SS- represents seconds.

You can then use StyleVision to create a style sheet which formats the date in a way suitable to your target audience. StyleVision is a graphical stylesheet design tool that allows drag-and-drop design of XSLT and XSL:FO stylesheets to render XML data in HTML, Microsoft Word, PDF, and other formats. To use the date formatting functionality within StyleVision:

  • Select the contents placeholder or input field of the node.
  • In the Properties sidebar, select the content item, and then the Content group of properties.
  • Click the Edit button of the Input Formatting property.
  • The Input Formatting dialog will appear:

StyleVision date formatting

  • Select the Formated radio button. This will allow you to choose which data type you would like to use, and if you have selected a date, you can then choose the format for the date.

You can also select other date and time formats here. I would strongly recommend using the date picker. In order to insert the date picker, the cursor must be between an xs:date or xs:dateTime node. You then go to Insert on the main menu and Select Insert Date Picker. If the cursor is not between xs:date or xs:dateTime node the Insert Date Picker menu item will be greyed out.

Numbers

Decimals can be preceded by either a point or a comma depending on the locale. There are also differences for how thousands are represented. StyleVision provides functionality where you can format a number for your intended audience:

  • Select the contents placeholder or input field of the node.
  • In the Properties sidebar, select the content item, and then the Content group of properties.
  • Click the Edit button of the Input Formatting property.
  • The Input Formatting dialog will appear

StyleVision number formatting

  • Select the Formatted radio button. This will allow you to choose the number format.

Money

The issues involving numbers also apply to money, but in addition to this there are different conventions for representing the currency symbol. Some currencies share the same name and symbol, such as the dollar, but the Australian, Canadian and Singaporean dollar are not the same currency, and this should be identifiable. You can deal with the numbers as shown above, but the issue of whether the currency name or symbol should go before or after the number is likely to be dealt with as part of the translation process.

Address

One of the problems faced by customers buying from a foreign company while making an online purchase is that the system does not allow them to enter their address properly. There are many differences, such as the house number being before or after the street name, the order the components of the address are placed and the format of the zip/postal code. CEN (The European Standards Institution) has developed a standard which lists the components of an address, and the UPU (Universal Postal Union) is further developing this to produce a comprehensive list of name and address elements. I would recommend that you ensure that you are getting the data you need for your main target markets but make sure that someone from another country can also enter their address. A drop-down list of countries could be used to ensure that there is error checking when you know certain components of an address are required but does not produce the error for other countries where you do not know the address structure.

Credit Cards

Some US-based web sites will not accept credit cards from outside the US. As a security check they insist on a valid US address. If you want to accept credit card payments and do business with people outside your country, you should check that foreign credit cards will be accepted.

RTL (bidi)

In many languages the text is being read from left to right but this is by no means universal. Arabic and Hebrew are written from right to left. In XML documents this causes further confusion as the XML elements are read from left to right but any text should be read from right to left. The ITS namespace has a direction attribute which can be used to identify which direction should be read. <its:span dir="rtl">متعة الأسماك!</its:span>

Sorting

There are differences in how alphabets are sorted. Some Scandinavian languages have an ‘aa’ character which is usually, but not always, sorted at the end of the alphabet. If you have set the language in your XML document and use xsl:sort for your XSL document then the sorting should work according to the sorting rules for that language. However, you should check that your processor does this as that is not always the case. The example files which come with StyleVision contain examples for sorting. Select StyleVision examples, then the tutorial folder, then sorting and open the file SortingOnTwoTextKeys.sps. To see how the sorting works go to the design view and right click on the member element. Then select the ‘sort by’ option on the context menu. Here you can control how the sorting works for this particular list.

Exclamation and Question Marks

In English, questions and exclamation marks are always at the end of the sentence, while in Spanish this punctuation occurs at the beginning and end of a sentence. This is something which will usually be corrected during the translation process.

Conclusions

Internationalization is an important step in ensuring the widest target audience for your application, and that translation is as cost effect and easy as possible. Your approach to this should be very pragmatic. Time spent up-front sorting out internationalization will result in huge benefits throughout the process and significantly increase marketing potential for your product. The purpose of this article was to present an overview and introduce you to internationalization. There is a lot more useful information available in the references listed below. Tools such as XMLSpy and StyleVision, both of which are included in the Altova MissionKit software suite, go a long way in making the internationalization process for XML documents much easier by providing a lot of in-built support for internationalization. The Internationalization Tag Set from W3C is a very significant innovation which is a great addition to the toolkit available to a developer who wants to build internationalized XML applications. XML is a technology which has had internationalization and translation in mind since its inception. The use of Unicode as the default encoding for XML is very significant and greatly facilitates dealing with any internationalization problems you may come across. The functionality available within the Altova MissionKit, ITS and Unicode are the basis for creating good internationalized applications.   Reference The following is a list of useful web sites and other resources providing further information on internationalization: Leading XML tools provider – Altova https://www.altova.com/ . They also offer a free trial of the MissionKit: https://www.altova.com/download. Unicode web site http://www.unicode.org/ Internationalization Tag Set http://www.w3.org/TR/2007/REC-its-20070403/ W3C Best Practices for internationalization http://www.w3.org/TR/2007/WD-xml-i18n-bp-20070427/ Open Tag (Yves Savourel’s) http://www.opentag.com/ Yves Savourel, ‘XML Internationalization and Localization’, a book which is an excellent source of information. More information can be found at: http://www.opentag.com/xmli18nbook.htm The TM-Global research and resource web site publishes a lot of useful articles, opinions and surveys on translation, localization and industry standards http://www.tm-global.com/ Web sites of internationalization guru Tex Texin http://www.xencraft.com/ and http://www.i18nguy.com/ Localization Flow – web site of internationalization experts http://www.locflowtech.com/ Value for money XML-based TEnTs and translation tools are available from companies such as Kilgray Translation Technologies http://www.kilgray.com/

Tags: , , ,

Exploring Large XML/XBRL Documents with XMLSpy


Last week, while giving a demo of the new XBRL capabilities in the Altova MissionKit, we stumbled across an interesting question: What is the best way for a semi-technical SME (in this case a CPA) to navigate a large XML/XBRL document for data entry? XMLSpy, which is included in the MissionKit tool suite, has a lot of cool features and different views for XML data, including the ever-popular grid view for visualizing the hierarchical structure of an instance document in a graphical manner. The ability to easily expand and collapse containers and drag and drop to change position makes XMLSpy’s grid view a pretty good choice for the task.  XMLSpy grid view Of course let’s not forget that the XMLSpy XML editor also has a Find feature that would enable users to simply press Ctrl F or use the Find in Files window to find any element that they are looking for… but alas, in the case of XBRL, where element names are mindbogglingly verbose, this may be a challenge. Consider, for example, the US-GAAP’s aptly named <us-gaap:IncomeLossFromContinuingOperationsBeforeIncomeTaxesMinorityInterestAnd IncomeLossFromEquityMethodInvestments>. Not so much fun to type into a Find dialog… Our solution, therefore, and the winner for the easiest and most comprehensive way for even a non-technical user to find XML elements in a large document, utilizes a combination of longstanding XMLSpy features (the XPath Analyzer window) and a new feature in XMLSpy v2009, XPath auto-completion. Simply begin typing the element name in the XPath Analyzer window, and XMLSpy will show you all of the possibilities. Next, choose the one you are looking for, and XMLSpy will navigate directly to that node in the XML document.   xpath auto-completion in XMLSpy   Now that was easy! And better yet, you get to tell your friends that you know XPath. 😉 Of course, for developers, intelligent XPath auto-completion provides a lot more than the ability to find a node quickly. As you type, it provides you with valid XPath functions, as well as element and attribute names from the associated schema and XML instance(s). XMLSpy accounts for namespaces when listing options and even provides deep path suggestions when the required node is not in close proximity to the current context. XMLSpy is available standalone or as part of the award-winning MissionKit tool suite.

Tags: , , ,

What's New in XMLSpy 2009?


In addition to being tremendously useful, some of the new features in XMLSpy 2009 are just plain cool. The complete list of new functionality includes:

  • Support for XBRL 2.1 and XBRL Dimensions 1.0  
  • XBRL Taxonomy Editor
  • XPath auto-completion 
  • Native support for additional databases 
  • Support for XML fields in SQL Server
  • Extensions for identity constraints editing in Schema View 
  • Expanded source control system support
  • Support for the XSLT extension altova:evaluate  
  • Support for Apache FOP 0.95  

We’ve already blogged quite a bit about the first two items on the list: support for XBRL validation and XBRL taxonomy editing. Some more details on the other new features are below.

Intelligent XPath Auto-Completion

We’ve been delighted to receive feedback from customers who are really excited about this new feature. If you’re developing XSLT or XQuery, writing XPath expressions just got a lot easier. As you’re composing an XPath expression in Text View, Grid View, or the XPath Analyzer, XMLSpy now provides you with valid XPath functions, as well as element and attribute names from the associated schema and XML instance(s). XMLSpy’s intelligent XPath auto-completion accounts for namespaces when listing options and even provides deep path suggestions when the required node is not in close proximity to the current context. XPath auto-completion  

Native Support for Additional Databases

XMLSpy 2009 adds new native support for the latest versions of SQL Server and Oracle, and brand new support for PostgreSQL. Support for DBs in XMLSpy allows you to generate an XML Schema based on a database, import and export data based on database structures, and generate relational database structures from XML Schemas, and so on. The built-in Database Query window lets you perform queries against the database and edit the data. Here’s the complete list of databases with native support in XMLSpy:

  • Microsoft® SQL Server® 2000, 2005, 2008
  • IBM DB2® 8, 9
  • IBM DB2 for iSeries® v5.4
  • IBM DB2 for zSeries® 8, 9
  • Oracle® 9i, 10g, 11g
  • Sybase® 12
  • MySQL® 4, 5
  • PostgreSQL 8
  • Microsoft Access™ 2003, 2007

SQL Server support has also been enhanced to allow viewing and editing of XML fields that are stored in the database.

Extensions for Identity Constraint Editing in Schema View

Configuring identity constraints (i.e., key/keyref/unique values) is an important aspect of XML Schema development, especially for database users. Adding to existing support for editing these identity constraints, there are now enhanced visual cues and editing options in XMLSpy 2009. A new tab Identity Constraints tab in the Components entry helper window displays all existing constraints in a tree view and allows you to easily modify or create new relationships. Furthermore, identity constraints are now indicated by green lines, informative icons, and mouse-over messages in the Content Model View. A right-click menu allows you to easily add new relationships and specify field and selector values by typing them manually, using drop-down entry helpers, or by simply dragging and dropping the desired nodes. Schema identity constraints

Expanded Source Control System Support

Based on customer feedback, we’ve completely reworked the source control system interface in XMLSpy and also added the same level of source control support to UModel, our UML modeling tool, allowing both products to intelligently integrate with all major SCM tools. Once a project is bound to a version control system, XMLSpy automatically monitors the status of all files and prompts the you to check out a file whenever you starts to modify the document. In addition, the actual state of each file is shown through checkmarks or locks in the upper right corner of each file icon.   What do you think of these new features? What would you like to see added to the next version of XMLSpy? Let us know by commenting below.

Tags: , , , , ,

XBRL Support Added to Altova MissionKit 2009


xbrl_tools Earlier this week we blogged about the release of Version 2009 of Altova MissionKit, which includes the complete Altova product line. One of the major themes in this latest release is comprehensive support for XBRL across multiple Altova MissionKit tools, so lets look into XBRL itself and the new functionality in more detail.

What is XBRL?

The eXtensible Business Reporting Language (XBRL) is an XML-based vocabulary for electronic transmission of business and financial data. Currently in version 2.1, XBRL is an open standard that is maintained by XBRL International, a global non-profit consortium of over 550 major companies, organizations, and government agencies. XBRL was developed to facilitate business intelligence (BI) automation by enabling machine-to-machine communication and data processing for financial information with an eye towards cost reduction through the elimination of time consuming and error-prone human interaction. Official support from European Parliament and a mandate the United States Securities and Exchange Commission (SEC) has all but secured XBRL’s future as the official standard for financial reporting. You can learn a lot more about the nuts and bolts of XBRL in the XBRL white paper written by Altova’s Technical Marketing Manager, Liz Andrews.

XBRL Tools

All of the power and functionality that XBRL brings to financial data is useless without XBRL-conformant tools to interpret and process this data. In fact, understanding the importance of tools for XBRL, the XBRL recommendation writes software vendors into its abstract:

XBRL allows software vendors, programmers, intermediaries in the preparation and distribution process and end users who adopt it as a specification to enhance the creation, exchange, and comparison of business reporting information.

As such, XBRL development and data integration can be supported in a variety of different ways. The Altova MissionKit 2009 provides comprehensive support for working with XBRL, from validation and editing, to transformation and rendering, in multiple tools: XMLSpy 2009 – XBRL validator and XBRL taxonomy editor. When you need to ensure that your XBRL filing documents are valid and compliant, XMLSpy can be used to to validate an XBRL instance file against its corresponding taxonomy. If you need to extend a standard taxonomy to meet your organization’s filing needs, the graphical XBRL taxonomy editor in XMLSpy gives you a graphical model with tabs that organize different types of taxonomy elements, wizards, and entry helpers, all of which guide data input based on the structural information provided in the XBRL instance and base taxonomies. MapForce 2009XBRL exchange and data integration tool. When you need to extract financial data from back end systems and transform it into compliant XBRL filings, MapForce makes it easy with drag-and-drop, graphical data mapping and instant conversion. MapForce allows bi-directional data mapping between XBRL and databases, flat files, Excel 2007, XML, and Web services. Once your mapping project is defined, you can utilize it to automate quarterly and annual XBRL filing generation by transforming data with the MapForce UI, command line, or auto-generated, royalty free application code. This makes public financial data submission a repeatable and highly manageable process, allowing organizations to produce valid XBRL reports as required based on the variable data stored in accounting system fields – without having to define new mappings every quarter. mapforce_xbrl_thumb If you need to aggregate XBRL filings from different time periods or different organizations in your back end systems for analysis, MapForce lets you map the XBRL data to databases, flat files, XML – even Web services. Mapping data directly from its source format removes the need for re-keying and potential errors. Together, MapForce and XBRL enable the automation of the multi-dimensional financial analysis that organizations and stakeholders use to evaluate market, company, and industry performance on a regular basis. StyleVision 2009 – XBRL rendering tool. Once data is in XBRL format, and it’s time to render it for consumption on the Web or in printed materials, StyleVision lets you create a straightforward XBRL report to output the data in multiple formats. Simply drag and drop a taxonomy financial statement onto the design pane as an XBRL table, and then use StyleVision’s graphical interface to format stylesheets for simultaneous output in HTML, RTF, PDF, and Word 2007 (OOXML). The XBRL Table Wizard makes it easy to customize the table structure and specify the concepts to include in the report. In the case of the US-GAAP taxonomy, which provides, in addition to the hierarchical organization in its presentation linkbase, some best practices information on how to structure XBRL instances, you can simply select US-GAAP mode to have StyleVision automatically output the data according to this information. xbrl_design To try this powerful XBRL functionality for yourself, download a free, fully functional 30-day trial of the Altova MissionKit 2009, and please let us know what you think by commenting on this blog or in the Altova Discussion Forum.

Tags: , , , , , ,

Altova customer Recordare builds MusicXML-based solution


Case Studies

Recordare® is a technology company focused on providing software and services to the musical community. Their flagship products, the Dolet® plugin family, are platform-independent plugins for popular music notation programs, facilitating the seamless exchange and interaction of sheet music data files by leveraging MusicXML. Dolet acts as a high quality translator between the MusicXML data format and other applications, enabling users to work with these files on any conceivable system, including industry leading notation and musical composition applications Finale® and Sibelius®. The list of MusicXML adopters also includes optical scanning utilities like SharpEye or capella-scan, music sequencers like Cubase, and beyond. Dolet increases the MusicXML support in all of these programs and promotes interoperability and the sharing of musical scores. In creating the Dolet plugins, Recordare used Altova’s XML editor, XMLSpy, for editing and testing the necessary MusicXML XML Schemas and DTDs, and the diff/merge tool, DiffDog, for regression testing.

The Challenge

Music interchange between applications had traditionally been executed using the MIDI (Musical Instrument Digital Interface) file format, a message transfer protocol that has its roots in electronic music. MIDI is not an ideal transfer format for printed music, because it does not take into account the multitude of notations (e.g., rests, repeats, dynamics, lyrics, slurs, tempo marks, etc.) that convey much of the meaning. MusicXML is an open, XML-based file format specifically created to encapsulate musical notation or digital sheet music data that was built on top of previous formats, MuseData and Humdrum. XML lends MusicXML the power and flexibility to be easily accessed, parsed, rendered, and otherwise manipulated by a wide variety of automated tools, and its general acceptance as a standard makes it an ideal format for scoring using computer technology. Since its original release by Recordare in January of 2004 (version 2.0 was released in June 2007), MusicXML has gained acceptance in the music notation industry with support in over 100 leading products, and is recognized as the de facto XML standard for music notation interchange. These products would not have adopted MusicXML unless it could be used to exchange data with industry-leading applications like Finale and Sibelius. By developing advanced plugins for popular music notation suites, Recordare would be able to deliver to their customers all of the advantages that XML can bring for data exchange and standardization.

The Solution

Below is an example showing the score of the first few measures of Beethoven’s An die ferne Geliebte, Op. 98 as it is written in sheet music: and a small snippet of the same piece translated to MusicXML: The MusicXML-based Dolet 4 plugins for Finale and Sibelius provide a more accurate and usable representation of sheet music than Standard MIDI translation. For example, the images below show the same piece of music. On the left is a Finale 2009 rendering of a MIDI file exported from Sibelius, and on the right is the same application’s interpretation of a MusicXML 2.0 file exported from the same version of Sibelius.
In the MIDI rendition, vital information like chord symbols, lyrics, slurs, articulations, and even title and composer are omitted from the translation. In addition to providing native support for MusicXML, the recently released Dolet 4 for Finale and Dolet 4 for Sibelius plugins enhance the capabilities of these programs by adding advanced features like:

  • Batch translation
  • More accurate and reliable data exchange
  • More formatting control
  • Support for the MusicXML XML Schema (in addition to the DTD)

In developing the plugins, Recordare was subject to specific requirements dictated by the Sibelius and Finale applications. The Sibelius plugin was programmed in ManuScript, and is one of the largest plugins ever written in that language. Finale, on the other hand, requires plugins to have a C++ core, and Recordare implemented this, adding MusicXML logic in Java and a JNI layer to provide the two-way Java/C++ communication. Recordare’s Dolet plugins are now critical aspects of the music preparation process for many television and film scores as well as new music publications. Errors in translation need to be fixed in maintenance updates, while ensuring that no new errors are introduced into these complex translation plugins. Regression testing of the MusicXML file produced by the Dolet plugins is thus an essential part of Recordare’s quality assurance process. Recordare used Altova’s DiffDog in the development of the Dolet plugins. XMLSpy was used to test and edit their DTDs and XML Schemas, and DiffDog for regression testing the MusicXML files produced by the software. Recordare has several regression test suites covering a wide range of musical repertoire, from baroque to hip-hop. DiffDog allows easy differencing of multiple runs of these test suites, including the ability to ignore differences in XML metadata elements such as software version and XML creation date that always change across test cases. Recordare has used Altova’s XMLSpy XML editor to edit the MusicXML DTDs and XML Schemas, starting with the use of XMLSpy 3.5 (released in 2001) to create the earliest alpha and beta versions of the MusicXML DTD. Version 2.0 of MusicXML added a compressed zip version of the format, similar to what is used in other XML applications like Open Office and Open XML. XMLSpy 2008 Enterprise Edition’s comprehensive support for zipped XML files made it easy to test this new feature together with the Dolet for Finale plugin.

A small portion of the extensive MusicXML schema shown in XMLSpy’s graphical XML schema editor

XMLSpy’s support for XQuery has also contributed to Recordare’s regression testing efforts. In response to a customer request, Recordare now exports XML processing instructions from the Dolet for Sibelius plugin when it encounters a musical feature that it is unable to translate correctly. A simple XQuery execution to search for all the processing instructions in the XML files in a given folder lets Recordare check for the presence of these restrictions within each test suite, and then compare the resulting XML files using DiffDog between runs of the test suite. Recently, customer demand led Recordare to develop an XSD version of the MusicXML format. XMLSpy Enterprise Edition was used to develop and test the schemas. Schema validation, schema restriction and extension, and automatically-generated schema documentation were all able to be tested using XMLSpy’s features.

The Results

The Dolet plugins are extensions for common industry software that harness the built-in capabilities of the MusicXML format to make musical scores truly interchangeable across disparate systems and toolsets. These plugins have the capacity to render accurate and meaningful musical notation based on the powerful MusicXML specification. The leading XML Schema editing capabilities in XMLSpy and the strong XML and directory differencing support in DiffDog enabled Recordare to write and polish the MusicXML schemas and perform regression testing on the Dolet plugins. The resulting high quality of the schemas and software has made MusicXML and the Dolet plugins a key element of the toolkit for composers, arrangers, publishers, copyists, and engravers throughout the industry wherever printed music is used. Try XMLSpy, DiffDog, and the other Altova MissionKit tools for yourself with a free 30-day trial.

Tags: , , , , , , , , ,

Case Study: Equifax


equifax Check out the case study below to learn how leading US credit reporting entity Equifax® built an advanced SOAP interface for their identity verification and authentication Web service.

Overview

Equifax is a leading credit reporting entity and provider of analytical and decision support tools. Their real-time authentication system, eIDverifier, offers government and businesses personalized online security measures that help protect them against fraud and comply with federal legislation. The eIDverifier process is used within e-commerce and other online applications to authenticate users’ identities based on their answers to personalized questions drawn from Equifax’s extensive data stores. The authentication process consists of five steps:

  1. Integrity Check – eIDverifier standardizes and screens applicant-provided information to test for data inconsistencies and irregularities.
  2. Pattern Recognition – A pattern recognition algorithm is conducted on each transaction. For example, a velocity parameter determines the number of times an applicant has applied for authentication in a specific time frame.
  3. Identity Validation – To confirm an identity’s legitimacy, eIDverifier uses a “waterfall” approach in gathering validation information from multiple data sources. This means that if the identity cannot be validated with the first data source, eIDverifier will proceed to the next data source until the identity is validated.
  4. Interactive Query – eIDverifier presents multiple-choice questions to the applicant based upon “shared secret” information that should only be known to the applicant and Equifax. The question sets are customizable to meet individual risk thresholds.
  5. Decision Logic / Output Assessment – There are two output components to eIDverifier – an assessment score and reason codes. The assessment score indicates the likelihood of an applicant presenting fraudulent information, while reason codes provide important details on questionable information and highlight any discrepancies between the consumer’s application information and Equifax data sources.

eIDverifier relies on the SOAP protocol to send messages defining these interactions back and forth between the client interface and the Equifax servers. Third party institutions license the eIDverifier SOAP interface for use within their online application processes, enabling them to integrate its functionality and access information contained in Equifax’s databases.Equifax uses the XMLSpy XML Schema editor to graphically design the XSDs that serve as the foundation for their SOAP interface.

The Challenge

Equifax needed a sophisticated tool for designing the XML Schemas that would define the data types for their Web service, as well as a mechanism for creating the WSDL documents that would describe the interface as a whole. As a Java shop, Equifax needed a solution that would be compatible with their other development tools, and that would work seamlessly with the Eclipse IDE. Though there are plenty of Java tools available that have the capacity for XML Schema development, XMLSpy presented the most attractive option for schema design because of its comprehensive graphical design and editing options.The Equifax development team took a further step to simplify their Web services creation, using XML Beans and the Codehaus XFire/CXF Java SOAP framework to auto-generate WSDL from their XML Schemas.

The Solution

eIDverifier relies on a variety of different technologies to bring identity verification and authentication to its clients. XMLSpy provides the following benefits:XML Schema

XML Schema is used to express the structure of the data, as well as the individual elements and attributes that it is comprised of. Because a large portion of the data relies on end-user input in the form of address, phone number, driver’s license number, etc., it is vital that this information is in a format that can be digested by the system.Using XMLSpy’s graphical XML Schema editor, the Equifax development team was able to easily visualize and maintain the structure of their XML Schema. A portion of the schema that was created appears below:

SOAP interface

This data type definition provides the syntax, and dictates the structure, for the data that is transmitted by the eIDverifier Web service.

XMLSpy’s unique graphical XML Schema editor allowed the Equifax development team to create and maintain a complex schema definition without writing any code manually. They were also able to automatically generate human-readable documentation that can be used to present the architecture for review at any time in the development process, and that describes each element and attribute in detail.SOAP interface

WSDL

The processes executed by eIDverifier are described by a WSDL document that incorporates the XML Schema to provide information about data types, functions, and other interface details to the client – defining and dictating the actions taken by the client application to send and retrieve information between the end-user and the Equifax servers. The Equifax team chose to autogenerate a WSDL document using the Codehaus XFire/CXF framework. The XML Schema was used as the basis for an XMLBeans implementation, which was then compiled as a Java service class. Once the eIDverifier service was exposed, XFire automatically generated a WSDL – the WSDL is shown below in the XMLSpy graphical WSDL editor.

SOAP interfaceThis WSDL serves as the basis for the eIDverifier application, defining the ports and messages that make up the communication infrastructure of the Web service.

The Results

The eIDverifier SOAP interface allows external applications to access Equifax’s backend data stores, exposing it as a Web service and enabling them to retrieve secure information without jeopardizing the integrity of the Equifax mainframe. Utilizing WSDL and SOAP, and surrounded by Java architecture, eIDverifier is able to confirm user identity by returning a set of multiple choice questions based on the secure data maintained by Equifax.SOAP interfaceXMLSpy enabled the Equifax team to quickly and easily create a graphical schema representation and the matching documentation to serve as the basis for the Web service. It also allowed the development team to focus on their Java code, rather than the intricacies of XML Schema and WSDL design. The Altova MissionKit provides numerous tools for advanced Web services development, from the graphical XML Schema and WSDL editing discussed here, to SOAP debugging, and even graphical Web services generation and data mapping. Download a free trial to check it out for yourself.

Tags: , , , , ,

Case Study: Wrycan, Fitz & Floyd, MarketLive


wrycan Fitz and Floyd is a leader in design and manufacture of hand painted ceramic gift ware. In 2007, they approached Wrycan, an Altova partner focused on content-centric XML expertise and related software development, for help creating a solution that would allow Fitz and Floyd to interface their existing CRM system to their new Web-based storefront application from MarketLive, the leader in e-commerce software solutions. Fitz and Floyd had already purchased a license for the Altova MissionKit software suite, so Wrycan was able to jump right in and start mapping data from Fitz and Floyd’s Oracle database to MarketLive’s proprietary schema using Altova MapForce. Wrycan assigned the project to a Principal Consultant, who had plenty of previous experience with XML technologies (including XSLT and XML Schema) as well as with large-scale databases, but who had never before used MapForce, Altova’s data conversion, transformation, and integration tool.

The Challenge

Fitz and Floyd required a solution that would automatically synchronize data from their Oracle database to MarketLive’s storefront application. It needed to perform the following functions: inventory updates, product updates, and order status updates. This way, when a customer ordered a Fitz and Floyd product via the MarketLive interface, they would be getting real-time information about the company’s inventory. The solution needed to be simple to use, easy to maintain, cost effective, and completed on time, so they could put their new storefront into production promptly. Fitz and Floyd’s existing data was housed in an Oracle 8.0.5 database and was organized according to internal requirements. In order to transform their data into a format that would work with MarketLive’s storefront application, Fitz and Floyd’s data needed to be mapped to MarketLive’s XML Schema. In addition, there needed to be a system in place to track and log any transaction errors that occurred.

The Solution

Because of MapForce’s ease-of-use, the Principal Consultant was able to get started using its intuitive features right away. Wrycan used MapForce to map the transformation from Fitz and Floyd’s Oracle database to the XML Schema definition (XSD) instance provided by MarketLive. Using the database as the source component and the XSD as the target, the following mapping was produced: MapForce mapping transparent In order to map to some XML Schema entities that were not explicitly defined in the original MarketLive schema, Wrycan used Altova XMLSpy’s graphical XML Schema editor to fill in the gaps, adding attributes to the schema that had not previously existed and thus ensuring that all necessary Fitz and Floyd data would be mapped to the MarketLive Web interface. An example of the schema modifications is shown below: XML Schema modifications Wrycan used MapForce’s unique code generation capabilities to automatically produce a Java applet that was used to update Fitz and Floyd’s product, inventory, shipping, and order status information programmatically. This specialized applet was then packaged along with Wrycan’s proprietary Transaction Manager. MapForce made it very easy to update and redeploy the data mapping requirements as they changed throughout the project. Because of MapForce’s ease of use and built-in code-generation capabilities, less technical users can also update the data mapping when there are changes.

Simple Web-based Transaction Manager

Utilizing open source Java technologies such as Apache Tomcat and Quartz Enterprise Job Scheduler, Wrycan was able to create a simple transaction manager that allowed the transactions handled by the MapForce-generated, Java-based data integration applet to be scheduled, processed, and logged. The Transaction Manager is a custom software application made specifically for Fitz and Floyd by Wrycan, but built in such a way that it can be reused for future clients. It consists of several components:

  • User interface – allows the integration of MapForce-generated Java code
  • FTP interface – adds the ability for files to be downloaded for transformation from Oracle database format to the eCommerce platform XML format or vice versa
  • Scheduler – allows the automation of the data migration
  • Reporter – stores transaction results in XML files accessible in the user interface and also has the ability send emails in case of exceptions

The Transaction Manager’s user interface is the point of contact for Fitz and Floyd to control and schedule any data transformations. Because Wrycan wanted to be able to reuse the Transaction Manager, they chose to generate the MapForce code in Java, a platform-independent programming language. (MapForce can also generate application source code in C# and C++.) This code is an integral part of the Transaction Manager, as it dictates the data mapping process, allowing Fitz and Floyd’s internal information to be accessed via the MarketLive interface. The FTP interface is a simple way to manage the transfer and delivery of files from within the Transaction Manager once the MapForce-generated Java applet has transformed the data according to the MarketLive schema. A built-in batch scheduler allows Fitz and Floyd to automate the data migration operations by content type (i.e. order, inventory, product, etc.). Batch jobs The reporting component allows the result of each transaction to be logged in XML. Because of this, if any transaction errors occurred, Wrycan was able to use Altova XMLSpy to analyze and debug the issues.

The Results

Fitz and Floyd now has an easy to use data integration layer that is extensible by adding new MapForce transformations, and they can easily adjust their current transactions. Any updates made to the Fitz and Floyd Oracle database are automatically transferred to the MarketLive application in a format that it can readily understand. Log Details Because the Transaction Manager application is based on platform-independent Java code (generated by MapForce), Wrycan also has a reusable application that can be used as an asset by any online retail company. Wrycan is now able to approach potential clients with a proven data integration layer product that provides job scheduling, email notification, and FTP integration and can utilize any database or schema output via a custom Altova MapForce transformation. When speaking about this project, Dan Ochs, the principal consultant at Wrycan involved with the Fitz and Floyd application stated “MapForce has proven to be an easy-to-use, effective tool for making the data integration and mapping process much easier and faster to implement.” This and many other customer case studies involving Altova solutions are available in the Altova library.

Tags: , , , , ,