XML Schema Archives

Tag Archive for: XML Schema

Benefits of an XML Schema Manager

November 9, 2022/in Tools, XML /by Erin Cavanaugh

Developers working with XML often need to deal with multiple DTDs and XSDs that define industry-standard vocabularies. Whether it’s DITA for technical writing, HL7 for healthcare data, CbCR for financial reporting or any number of examples, it becomes a challenge to manage the various schemas—and numerous versions thereof—on a day-to-day basis.

For flexibility and convenience, all Altova XML-enabled products include its XML Schema Manager. This provides a centralized utility that makes it easy to download and manage industry schemas for use across the product line. Let’s see how it works.

Tags: XML Schema, XSD

XML to XSD: Generating a Schema from Multiple XML Instances

August 25, 2021/in Development, Tools, XML /by Erin Cavanaugh

I was recently talking with one of our Support Engineers about common questions he receives from customers. One that comes up a lot is schema generation, specifically, “How can I generate an XSD from XML?” The answer, of course, is to use your favorite XML editor. XMLSpy will generate a valid XSD from a DTD, a relational database structure, a JSON Schema, and of course, an XML instance.

But what many customers are surprised to learn is that the XML editor will also create an XML Schema based on a group of related XML files – and this is a common requirement. Let’s take a look at how it works.

Tags: XML Editor, XML Schema, XMLSpy, XSD

What’s New in XML Schema 1.1?

March 21, 2016/in Development, Tools, XML /by Erin Cavanaugh

Since the introduction of XML Schema 1.0, the standard has been widely adopted. Developers have requested several changes over the years, most notably the ability to specify and validate more granular business rules and also to allow flexible schema evolution. There are numerous changes and enhancements in the XML Schema 1.1 Recommendation aimed at addressing these issues.

Let’s take a look at the most notable new features and some resources for learning more or getting started right away with XML Schema 1.1.

shutterstock_137938508

XML Wildcard Data Mapping and Transformation in MapForce

December 17, 2013/in Data Integration, XML /by David McGahey

MapForce 2014 now supports mapping of XML wildcards that create a designed-in extension mechanism for XML Schemas, implemented as <xs:any> or <xs:anyAttribute>.

Consider this XML fragment viewed in XMLSpy:

Fragment of an XML document following the GPX XML Schema

This small portion of a very long file shows two track points recorded during a road trip by a Garmin GPS device. The XML data follows the GPX XML Schema, with Garmin extensions to add speed and course heading values. XML wildcard mapping lets you map these extensions in MapForce just like any other elements explicitly defined in the XML Schema.

Let’s assume we want to read the .gpx file and extract the highest speed recorded for a trip:

Output of the completed mapping Read more…

Tags: data mapping, MapForce, XML Schema, XMLSpy

Learn About XML Schema 1.1

November 5, 2013/in Technology /by Erin Cavanaugh

Coinciding with support for XML Schema 1.1 validation and editing in XMLSpy 2014, we’ve released a brand new Altova Online Training course on XML Schema 1.1 technology. The course covers the fundamentals of the XML Schema language and is appropriate for beginners as well as experienced XSD developers who want to learn the ins and outs of the latest version of the standard.

This self-paced, online course is broken into nine chapters, which you may complete in order or simply focus on the areas that interest you:

Schema Languages
Schema Core Concepts
Built-in Datatypes
User-defined Simple Types
Complex Types
Namespaces
Identity Constraints
Include / Import / Override
New Features in XML Schema 1.1

The XML Schema 1.1 Technology course provides a balanced mix of narrated content, hands-on practice exercises, and video demos, helping you learn about XML Schema in your preferred style.
Like all Altova Online Training courses, the XML Schema 1.1 training is provided 100% free of charge. Simply access the class on our training page and complete it at your own pace.

What course would you like to see next? Leave us a comment here on the blog!

Tags: Altova Online Training, v2014, XML Schema, XML Schema 1.1

Use XPath Expressions to Refine Data Selection

September 12, 2013/1 Comment/in Data Integration, XPath /by David McGahey

In this era of big data we often need to simplify analysis and communication by creating customized views into sections of a larger file. XPath, short for XML Path Language, is designed to allow users to address parts of large XML documents. XMLSpy supports XPath with an XPath Analyzer window and in its interface for charting XML data, MapForce supports XPath functions for XML data mapping, and StyleVision supports XPath in conditional templates, extension templates, and template filtering.

Announcing RaptorXML, the Hyper-fast XML and XBRL Server

April 16, 2013/in XBRL, XML, XQuery /by Erin Cavanaugh

We are very excited to be at the XBRL 26 Conference in Dublin, Ireland today to announce a brand new server product in the Altova family of XML and XBRL tools! Altova RaptorXML is a hyper-fast XML and XBRL validation and processing server. It’s Altova’s third-generation XML and XBRL engine, built completely from scratch to help organizations efficiently validate, process, transform, and query the vast and ever-increasing amounts XML and XBRL data being generated as a result of XBRL compliance regulations and myriad other big data trends. RaptorXML is written to be highly scalable for today’s multi-CPU and multi- core computers and servers. This, along with high performance code optimizations and an extremely low memory footprint, has helped make RaptorXML a lightning-fast XML and XBRL server that can meet the demands of today’s data processing applications. Simply put: we architected RaptorXML to combine the performance benefits afforded by modern parallel computing environments with strict compliance to the latest versions of all relevant XML and XBRL standards. RaptorXML includes support for the very latest versions of all relevant standards and has been submitted to rigorous regression and conformance testing. The server will be available in two versions, both of which are available for Windows, Linux, and MacOS platforms. RaptorXML Server supports validation and processing of:

XML 1.0 & 1.1
XInclude 1.0
Xlink 1.0
XML Schema 1.0 & 1.1
XPath 1.0, 2.0 & 3.0
XSLT 1.0, 2.0 & 3.0 (subset)
XQuery 1.0 & 3.0
And more

RaptorXML+XBRL Server supports all the features of RaptorXML Server, with the addition of processing and validating the XBRL family of standards:

XBRL 2.1
XBRL Dimensions
XBRL Formula 1.0
XBRL Functions
XBRL Definition Links

Developers creating solutions using Altova MissionKit XML development and XBRL development tools will be able to power their server applications with RaptorXML for hyper-performance, increased throughput, and efficient memory utilization, giving them the opportunity to validate and process large amounts of XML or XBRL data cost-effectively. Check out the complete list of supported XML and XBRL standards and more details on this groundbreaking new server product. RaptorXML will be available to download and purchase in May.

Tags: RaptorXML, XBRL, XML, XML Schema, XQuery, XSD, XSLT

New! XSLT Technology Training

February 27, 2012/in XPath /by Erin Cavanaugh

We’re excited to introduce our new XSLT Technology training as the latest Altova Online Training offering. As with all our trainings, XSLT Technology is released as a free, self-paced course, available online, so students can fit it into their busy schedules.

XSLT transforms XML data into other formats, and this course will transform a beginner XML student into an advanced user. Intermediate and advanced students will gain valuable techniques to add to their XML toolkits.

Processing the Groupon API with MapForce – Part 2

August 16, 2011/6 Comments/in Data Integration /by David McGahey

In Part 1 of this series we described how to connect Altova MapForce to the Groupon API. We queried the API for a list of Groupon divisions, then used the list to create API queries for all the current deals from every division. In this part, we will execute the /deals queries and filter the response for the most interesting data. The list of /deals queries we built previously looks like this: To process all the queries, we can connect the list as a dynamic file input to a new mapping component. When we needed a new component last time, we dropped an API /divisions query into the mapping, and let MapForce create an XML Schema automatically. We could do the same thing here by dropping in an API /deals query as an XML input file. There’s just one small issue — although the Groupon API online documentation clearly describes the queries we can make, it is vague about the information that will be returned. Before we send dozens of queries to the API for all the current deals, we probably want to know a little more about the data that will come back.

Let’s Make a Deal Like Yogi Berra said, you can observe a lot just by looking. Let’s start by running a /deals query in XMLSpy. That will let us examine the response to a query for one division before we pull in a potentially unwieldy volume of data. The XMLSpy File / Open menu includes the same Switch to URL option we used in MapForce in the earlier post. If we enter the /deals API query for a division that covers a large metro area – say Dallas – we are likely to get enough deals instances to extrapolate the characteristics of the entire data set. XMLSpy opens the response to the /deals API query in Text view just as if we opened a local file: As expected, we got quite a bit of data when we requested all the deals for a single division! A fast way to analyze the structure of this data is to use the XMLSpy DTD / Schema menu option to generate an .xsd file from the xml. Shown below is a reduced view of the entire generated .xsd file based on the response to the /deals query for Dallas: We can dig even deeper, following Yogi’s advice like déjà vu all over again. Expanding all the elements to review the XML Schema reveals some curious anomalies. For instance, there are two elements named redemptionLocation with different definitions. The first contains a sequence of child elements: And the second is defined as a simple string: Going back into the xml data for Dallas and searching for redemptionLocation displays these examples: And: And: Now this is really interesting, because redemptionLocation = ”online” identifies deals that can be redeemed from anywhere, instead of by a visit to a bricks and mortar location in the division where they are advertised. What if we ran the /deals API queries for all divisions and extracted a list of all the online deals? That would be one extreme Groupon! Only Ask for What You Need The Groupon /deals API query supports an optional parameter called &show= that allows users to limit the data returned. Applying this parameter can save bandwidth and reduce processing time for the data transformation by removing unwanted data from the API response. We can also simplify our final result by including only the most interesting information, including the link to the Groupon web page for each deal. After we remove unwanted elements from the generated Dallas schema, our final version for the summary of online deals looks like this: When we add the &show= parameter to our MapForce mapping to request only the elements included in the simplified XML Schema, the queries look like this: Now we can drop the revised .xsd file into the mapping and connect the list of API /deals queries as dynamic input. We don’t need to delete the text file we used to collect the list of queries — that might continue to be helpful for future debugging. These changes complete the input side of the data mapping. Defining the Data Transformation Output Back in XMLSpy we can make a couple more revisions to the input XML Schema to design a new version for output: We discarded the response element since it doesn’t add any value, and eliminated the redemptionLocation element that we don’t intend to include in the output. We also added a date element for a timestamp, because our output file will be a snapshot of data that is constantly changing. After saving this version of the .xsd file in XMLSpy, we can drop it into the MapForce mapping. Shown below is the output side of the mapping with the output component partially connected. The filter at the top reads the redemptionLocation element to select only online deals and the now function inserts the date: The last revision we made in the output XML Schema was to change several element types from dateTime, Boolean, and integer to the string data type to allow more descriptive text Here is the complete definition of the mapping with the final connections to the output component: Now for the Payoff When we click the Output button MapForce processes the entire mapping from beginning to end using the MapForce Built-in execution engine. Here’s a breakdown of the steps:

Run the /divisions query to get the current list of divisions
Concatenate strings to build the list of /deals queries for all divisions
Run the /deals queries to create dynamic data for the input component
Filter for online deals to generate the output component, execute the remaining mapping functions, and add the timestamp after all the deals are processed

MapForce takes only a few seconds to complete all those steps and generate an output file with a series of deals that look like this: In part 3 of this series we’ll design a stylesheet to automatically transform the XML output of our mapping into html for attractive presentation in a web browser and on mobile devices. See ya at the ballpark, Yogi! XMLSpy and MapForce are available together in the specially priced Altova MissionKit. See for yourself how easy it is to use the MissionKit to convert data from a Web API — download a free 30-day trial!
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: Altova XMLSpy, data integration, data mapping, MapForce, MissionKit, XML Editor, XML Schema, XMLSpy

Processing the Groupon API with Altova MapForce

August 2, 2011/2 Comments/in Data Integration, JSON /by David McGahey

We often think of a data integration project as a translation from one singular data input file to some other data set, but Altova MapForce lets you greatly expand the concept of an input file. For instance, the MergeMultipleFiles.mfd example installed with MapForce illustrates how you can use a filename with wildcard characters to merge multiple input files into a single output. A MapForce mapping input doesn’t even need to be a physical file – it can be a URL that returns predictable structured data, like the APIs for popular Web sites like Groupon and many others.

In this blog post we’ll describe how to use Altova tools to retrieve, filter, analyze, and present data available from a Web-based API, using Groupon as an example. If you want to follow along yourself, you will first need to visit http://www.groupon.com/pages/api to request your personal Groupon API client key. The Problem: All Deals Are Local The Groupon Web site and email subscriptions are great for finding deals in your local neighborhood, but what if you’re looking for a deal to use on an upcoming vacation, or for a gift for friends or family across the country? Sure, you could enter each location manually at the Groupon Web page, but that’s so last century. Let’s use the Altova MissionKit to automate things. The Groupon API offers two URL queries that return data in .json or .xml formats: the first returns a list of all Groupon localities (called divisions), and the second returns current deals information for one named division. If we want to see all the deals for more than one division, we need to resolve multiple URLs and aggregate the data into a single result. Yes, MapForce can do that! First We Need a Schema The Groupon API documentation describes the elements that will be returned by our requests, but doesn’t provide an XML Schema. That’s okay, we can use MapForce to generate one. All we have to do is open a new mapping design and choose Insert XML Schema/File, then click the Switch to URL button. Now we can enter the URL to retrieve the Groupon divisions list: When we click the Open button MapForce offers to generate the schema: When we click Yes, the File / Save dialog opens. I saved the schema as divisions.xsd, and the mapping with the new XML Schema inserted looks like this: And the Properties dialog for the XML Schema component automatically contains the API /divisions URL as the Input XML File: Check the Work We want to filter the Groupon divisions data to build a list of id names to use for deal queries for each locality. But before we go any further, now might be a good time to apply the text file trick from the Quick Solution for Complicated Functions blog post to look at the id values. When we insert the text file and connect the divisions and id schema elements, the mapping looks like this: We connected the division element to Rows in the text file in order to generate a new row in the text file for each unique division, so that Field1 in each row will hold the id. Clicking the Output button now generates this result: All we need to do is apply the concat string function to build the list of /deal URLs for all division IDs. The next step in the mapping looks like this: Rolling the cursor over the constant connected to value1 of the concat function displays its full definition: When we click the Output button to execute the mapping, the Output file now looks like this: As a further review, we can open the generated XML Schema in XMLSpy and display it in graphical Schema View: So far we have:

built a MapForce mapping that queries the Groupon API for all divisions
extracted the division id fields
and built a list of URLs for API queries to get the deals in each division

In the next post in this series we will process the list of deal queries as the input for a new mapping component and filter the output for some interesting information. Find out for yourself how easy it is to apply MapForce to convert data from a Web API! Download a free 30-day trial of MapForce.
Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: Altova XMLSpy, data integration, data mapping, MapForce, MissionKit, Web services, XML Schema

New XML Schema Editing Tools in XMLSpy

March 2, 2011/in Tools /by Erin Cavanaugh

Let’s take a look at some of the new features for working with XML Schemas in the latest release of XMLSpy.

Sorting in Schema View

When you’re working in the graphical XML Schema editor, you can now sort some or all of the schema components alphabetically with a single click. This is a huge time saver when you need to organize large schemas that have evolved over time and for understanding new schemas that come across your desk. Simply click the A-Z button in the Schema Overview window and select the sort options you prefer. Your components are instantly sorted in the graphical view… …and when you click over to Text View, the schema code itself is reordered and organized.

Schema Refactoring

Another new feature that’s invaluable when working with complex schemas or ones inherited from other developers is schema refactoring support. Taking advantage of XMLSpy’s powerful Find in Schemas functionality, you can quickly locate all the instances of a global element or type across a schema – and all imported and included schemas – and then rename that component in each place it occurs. This makes schema refactoring easy while ensuring the validity of all impacted schemas.

Intelligent Support for Changing Types

You may also find the need to change the type of an element or a base type, which previously meant reconfiguring all the facets associated with that component. This new feature gives you the option to preserve any facets or attributes that are compatible with the new type. Rename components to refactor schema

Customizable XML Schema Documentation

Another option schema developers have requested quite often is the ability to customize the comprehensive XML Schema documentation generated by XMLSpy. This is now possible via integration with StyleVision. The StyleVision stylesheet design tool provides countless advanced options for customizing the documentation templates, from adding your company’s logo and branding to changing the appearance and organization of the documentation completely. StyleVision integration also gives you the option to generate documentation in PDF in addition to the HTML, Word, and RTF output options available for the fixed documentation in XMLSpy. Here’s an an example of XML Schema documentation we customized for the fictional Nanonull corporation: Embed image in XML XMLSpy 2011r2 also ships with several useful documentation templates that can be used as is or customized further in StyleVision:

XML Schema Structure Overview: documents the structure of global elements and complex types to the number of levels you specify
All XML Schema Type Derivations: displays all simple and complex types and their derived types in a tree format
XPath List: generates all possible XPath statements that are possible for a schema based on a user-specified number of levels
Statistics: lists the numbers of elements, types, attributes, groups, etc., for the overall schema, and for each associated schema file

To use a custom schema documentation template, you must have XMLSpy and StyleVision installed. If you are a MissionKit customer, you already have both tools and can take advantage of this feature and countless other options for working with XML Schema, from stylesheet and report design, to data mapping, and more. If you aren’t currently a StyleVision customer, grab a free trial from our Web site. The XBRL and WSDL documentation generated by XMLSpy may also be customized using StyleVision. As always, we rely on your input and feature requests when planning each release – please let us know what you think of these new features and what you’d like to see in the next product version by leaving a comment here on the blog.

Tags: v2011r2, XML Editor, XML Schema, XMLSpy

Updated Industry Schema Library

May 26, 2010/in XBRL /by Niki Devgood

“The nice thing about standards is that there are so many to choose from.” – Andrew S. Tanenbaum (attr.)

Maybe we can help. Altova’s updated schema library is a collection of over 100 industry and cross-industry XML Schema and DTD specifications – all in one central location. XML standards (and technology standards in general) are used to promote information sharing and interoperability across disparate software and systems. In a perfect world, this would translate to a global network of data being exchanged seamlessly between information partners… in a perfect world. However, there are some industries that are reaping the rewards of standardization, and hopefully these efforts will pave the way for more. Very recently we have seen an increased interest and even some actionable mandates in XBRL for financial data, HL7 for exchanging healthcare messages, and NIEM for inter-agency communication within the United States government. Of course, if you do find yourself working with some conflicting technology standards, you should probably take a look at Altova MapForce. With native support for visually mapping pretty much any data formats you can think of (XML, databases, flat files, EDI, Excel 2007+, XBRL, and Web services), easily adding data processing functions, and a whole bunch of automation options, you may even find that creating data integration solutions is well… kind of fun. Check out Altova’s Industry Schema Library, or download a free 30-day trial of MapForce today!

Tags: Altova, data mapping, EDI, HL7, NIEM, XBRL, XML Schema

Altova Adds to NIEM Support in v2010r3

May 10, 2010/2 Comments/in Technology /by Niki Devgood

Clashes between government agencies are the stuff of legends and entire TV shows: the FBI and the CIA, the local sheriff’s department and the big city CSI, Homeland Security and the Pentagon, Jack Bauer and CTU. Perhaps this has to do with the territorial nature of some of these entities, but perhaps, just perhaps, technology has a part to play in these conflicts. Incompatible architectures and data formats, legacy systems, and other technology road blocks within the United States government sometimes made it difficult for agencies to efficiently share pertinent information such as arrest reports, amber alerts, immigration details, and more.
The National Information Exchange Model (NIEM) is an XML-based standard for data exchange between United States government agencies and their information partners. It aims to provide a means for these entities to improve decision making, achieve greater operating efficiency, mitigate risk, improve public safety, and increase ROI through intergovernmental information exchange.
As a leader in XML tools and technologies, the Altova MissionKit already inherently supports NIEM on a variety of different levels. In v2010r3, we added two very specific features to XMLSpy that extend the XML editor with capabilities that are invaluable for NIEM development. For more on these features, see the recent NIEM blog post by our CEO and XML Aficionado, Alexander Falk.
Below is a brief overview of the support provided for NIEM across the MissionKit, from XML Schema editing in XMLSpy, to XMI import and UML diagramming in UModel, to data integration in MapForce, and beyond.

XMLSpy

XMLSpy® 2010 provides a graphical schema design interface which simplifies the development of XML Schemas, allowing XML code to be generated in real-time behind the scenes. Users can switch back and forth between this view and text view at any time during their design process.

The graphical schema editor provides built-in support for creating the IEPD Extension Schema, Constraint Schema, and Exchange Schema.

In addition to the broad XML Schema support described above, XMLSpy provides comprehensive support for all other prevalent XML technologies, including:

Graphical and text-based editing views
Sophisticated XML Schema and XML instance validation
XML instance document creation
XSLT editor, debugger, and profiler
Web services tools (WSDL editor, SOAP client, SOAP debugger)

UML Diagrams

The first stage in any development lifecycle is a thorough business requirements review. UML is particularly well-suited to modeling software projects because it provides a standardized approach to the design process. Scenario-based planning is recommended for NIEM, and though there is no UML diagram requirement, class diagrams, use-case diagrams, and sequence diagrams all provide value to IEPD documentation. In addition, NIEM provides a free tool for mapping data requirements based on an uploaded XMI representation of a UML model
UModel® 2010 is an advanced UML tool that supports all fourteen UML 2.3 diagrams, BPMN, SysML, and more. It works seamlessly with XML technologies like XMI, letting users easily import and export XMI based on their UML diagrams. UModel can even render XML Schemas in a format similar to UML diagrams and integrates seamlessly with all other MissionKit tools including XMLSpy, MapForce, and SchemaAgent. UModel also supports code generation, round-trip engineering, and reverse engineering, as well as integration with the popular Visual Studio and Eclipse IDEs.
UModel can be used two different phases of NIEM development:

Scenario Planning – to model and document with use case diagrams, sequence diagrams, and business process modeling
Map & Model – to develop class diagrams to create the exchange content model

XML Schema Management

Because of the potential to have many different schemas in each IEPD, XML Schema management can become an arduous process. SchemaAgent® 2010 is a unique file management tool that lets users visually manage their XML Schema, XML, XSLT, and WSDL assets in a project-based environment so that changes and structure of the file configuration can be easily recognized. SchemaAgent users can even view mapping files created using MapForce.

Data Mapping & Exchange

Data mapping is a crucial step at several levels in the NIEM process. Mapping helps users generate a subset schema to determine what elements can be used from the core and domain models, it is also used to perform a transform from one NIEM vocabulary to another. Some NIEM implementations also employ Web services to provide a mechanism for the exchange.
MapForce® 2010 is an any-to-any data mapping and integration tool that supports all of the potential data formats used in the NIEM lifecycle including XML, databases, flat files, Excel 2007+, and Web services. MapForce maps data based on its underlying structure (i.e. XML Schema), enabling mapping designs to be reused for recurrent transformations. MapForce also supports mapping to and from multiple source or target components. In addition, MapForce can be used to generate documentation detailing mapping projects for non-technical stakeholders in a variety of formats such as HTML and Microsoft® Word.

To try out this functionality on your own, download a free 30-day trial of Altova MissionKit now!

Tags: MapForce, MissionKit, NIEM, SchemaAgent, UModel, XML Editor, XML Schema, XMLSpy

A Developer’s Life is Never Simple

December 22, 2009/in /by David McGahey

Earlier this month in our blog post on Comparing XML Schemas we showed a realistic – but simplified – example to illustrate a slick new feature of DiffDog 2010 to compare XML Schemas and update corresponding XML data files by generating XSL transformations.
A real-life XML developer’s project is rarely as small and straightforward as the example we used. In this post we will take a look at some typical complications developers face every day and how the Altova MissionKit cuts through complexity to enhance productivity.
Longer XML Schemas and Larger Mappings
Your XML Schemas are not likely to be as short and simple as the ones in our earlier post, and you will likely want to save your work while you are mapping your XML Schema migration. DiffDog lets you save your mapping in an XML Schema comparison file that you can reload later to continue your work, or to share with a colleague.

If you need to compare two XML Schemas on a regular basis, the XML Schema Comparison Document can be a valuable time saver.
Embedded XML Schema Assignment
The screen shot below shows the XML data file from our original example with one important difference. In this version the story element includes an embedded reference to the XML Schema on line 2.

When we transform this file with the simple XSLT we created in DiffDog, the new XML data file will not include the updated XML Schema reference. In our earlier blog post embedded XML Schema references weren’t an issue because we used the XMLSpy Project / Properties menu option to assign default XML Schemas for each folder.
If there are many XML data files to transform, and they require embedded XML Schema references, we can take advantage of an additional feature of MapForce. We can export our mapping from DiffDog to MapForce, as we did in the earlier post, and then use the Component Settings dialog for the MapForce output component to include the XML Schema reference.

Document Your Work
Developers can lose productivity trying to retrace history when a project needs additional work months or even years after an early iteration. The MapForce Generate Documentation feature can help us avoid this frustrating experience.
MapForce will document the mapping of each element in the XML Schemas in Microsoft Word, RTF, or HTML formats. Regardless which format we choose, the resulting document is an excellent stand-alone project artifact, or it can be further edited and included in a larger report.

We can even combine the MapForce mapping documentation with full descriptions of each version of the XML Schema generated with the XMLSpy XML Schema Editor documentation feature.
Source/Version Control
Source/version control systems let teams of developers work closely together on the same project without a risk of overwriting each others’ changes. Because a version of the source code is saved at each stage of the design process, it is very easy to look at or revert to an earlier version when needed.
Altova has implemented the Microsoft Source Code Control Interface (MSSCCI) v1.1 – v1.3 in XMLSpy and tested support for many popular source control systems, so we can manage the files in our XML Schema evolution project across the development enterprise. Additionally, DiffDog can be integrated with source control systems as the default comparison tool. DiffDog can even generate differences report files in a variety of formats.
See for yourself how the tools in the Altova MissionKit can cut through the complexity of your own XML, Web Services, data integration, XML publishing, XBRL, and UML modeling development projects – download a free 30-day trial!

Tags: Altova XMLSpy, diff merge tool, DiffDog, MapForce, MissionKit, v2010, XML Schema, XMLSpy, XSLT

Comparing XML Schemas with DiffDog 2010

December 10, 2009/in /by Erin Cavanaugh

DiffDog 2010 includes a powerful new tool to compare XML Schemas that XML developers and others can use to update existing XML data files as XML Schemas evolve. This post takes a look at an example scenario for this feature.Before we drop into the new functionality, let’s take a quick look at two XML Schemas using the DiffDog File Compare feature. Of course, just like in previous versions, DiffDog 2010 users can compare XML Schemas as .xsd documents and display differences in a color-coded, XML-aware format. This is a good way to identify and manage differences in XML Schemas, especially when you want to review revisions to industry-standard XML Schemas that evolve over time.What’s new in DiffDog 2010 is an additional XML Schema Differencing option that graphically displays two XML Schemas side by side, identifies identical elements automatically, and lets users map differences and generate XSL transformations to update XML data files.Here’s our first view when we open the same two XML Schemas shown in the file comparison above, using the new XML Schema Differencing feature. The root elements of the two XML Schemas are automatically connected. We can click the Compare button in the toolbar to automatically connect identical elements in the two XML Schemas. (Of course we could also select Compare XML Schemas from the right click context menu, or choose Start Comparison from the Diff and Merge menu, or press the F5 keyboard shortcut – DiffDog gives you many options to perform the same task, so you can work the way you like.)Next, we can map elements with different names in the two XML Schemas by manually connecting the pointer arrows between them. In this example most of the changes to the version of the XML Schema on the right simply give elements new names that will be more clear when the XML Schema and its data files are distributed through our enterprise. When all the elements are mapped, we can generate an XSLT file to transform existing XML data files based on the XML Schema on the left to reflect revisions in the newer version on the right. This feature is designed to rescue XML developers from the tedious tasks of writing and debugging XSL transformations by hand. Here is an example of an original XML data file based on the XML Schema on the left side, as viewed in Altova XMLSpy: The output file after applying the XSL transformation we created with DiffDog 2010 appears below. Note the substitution of the author element for writer, email for feedback, and so on. If there are many existing XML files that need to be transformed, the Project Management features of XMLSpy can help us automate the process. We can add external folders to an XMLSpy project. Using the XMLSpy properties dialog for each project folder, we can assign default values to assign an XML Schema for validation, the XSL transformation, and the destination of the output. Now we can select the input folder in the XMLSpy Project helper window and transform all the files in it with the single-keystroke F10 shortcut.When we originally mapped the XML Schema elements in DiffDog, we left the publication element on the left side unconnected, since it had no corresponding element in the earlier version of the schema. That means when we transform XML input files using the XSLT, the resulting output will not contain the publication element. If publication is a required element, we can call on Altova MapForce for a quick solution.One of the options in DiffDog is to generate a MapForce mapping rather than XSLT. When we choose this option, MapForce launches with our DiffDog mapping already loaded as a new MapForce design, as shown below. It’s easy to enhance the mapping by adding a constant as a default value for the publication element. Now we can save an XSL file from MapForce that reuses all the element mappings we originally designed in DiffDog and adds the constant. When we apply the new XSL to transform our original XML data file, we get a result that includes the default value for the publication element. This post started by describing the new XML Schema Comparison feature in DiffDog 2010. Fleshing out a simple – but typical – real-world example quickly highlighted additional tasks easily completed by taking advantage of tight integration with XMLSpy and MapForce.All three of these tools and more are available at substantial savings in the Altova MissionKit 2010, the integrated suite of XML, database, and UML tools designed to meet the diverse development and data management needs of today’s software architects and XML developers. Click here to download a free trial today!

Tags: Altova XMLSpy, diff merge tool, DiffDog, file comparison, MapForce, MissionKit, v2010, XML Schema, XMLSpy, XSLT

What's New in XMLSpy 2009?

February 25, 2009/in /by Erin Cavanaugh

In addition to being tremendously useful, some of the new features in XMLSpy 2009 are just plain cool. The complete list of new functionality includes:

Support for XBRL 2.1 and XBRL Dimensions 1.0
XBRL Taxonomy Editor
XPath auto-completion
Native support for additional databases
Support for XML fields in SQL Server
Extensions for identity constraints editing in Schema View
Expanded source control system support
Support for the XSLT extension altova:evaluate
Support for Apache FOP 0.95

We’ve already blogged quite a bit about the first two items on the list: support for XBRL validation and XBRL taxonomy editing. Some more details on the other new features are below.

Intelligent XPath Auto-Completion

We’ve been delighted to receive feedback from customers who are really excited about this new feature. If you’re developing XSLT or XQuery, writing XPath expressions just got a lot easier. As you’re composing an XPath expression in Text View, Grid View, or the XPath Analyzer, XMLSpy now provides you with valid XPath functions, as well as element and attribute names from the associated schema and XML instance(s). XMLSpy’s intelligent XPath auto-completion accounts for namespaces when listing options and even provides deep path suggestions when the required node is not in close proximity to the current context.

Native Support for Additional Databases

XMLSpy 2009 adds new native support for the latest versions of SQL Server and Oracle, and brand new support for PostgreSQL. Support for DBs in XMLSpy allows you to generate an XML Schema based on a database, import and export data based on database structures, and generate relational database structures from XML Schemas, and so on. The built-in Database Query window lets you perform queries against the database and edit the data. Here’s the complete list of databases with native support in XMLSpy:

Microsoft® SQL Server® 2000, 2005, 2008
IBM DB2® 8, 9
IBM DB2 for iSeries® v5.4
IBM DB2 for zSeries® 8, 9
Oracle® 9i, 10g, 11g
Sybase® 12
MySQL® 4, 5
PostgreSQL 8
Microsoft Access™ 2003, 2007

SQL Server support has also been enhanced to allow viewing and editing of XML fields that are stored in the database.

Extensions for Identity Constraint Editing in Schema View

Configuring identity constraints (i.e., key/keyref/unique values) is an important aspect of XML Schema development, especially for database users. Adding to existing support for editing these identity constraints, there are now enhanced visual cues and editing options in XMLSpy 2009. A new tab Identity Constraints tab in the Components entry helper window displays all existing constraints in a tree view and allows you to easily modify or create new relationships. Furthermore, identity constraints are now indicated by green lines, informative icons, and mouse-over messages in the Content Model View. A right-click menu allows you to easily add new relationships and specify field and selector values by typing them manually, using drop-down entry helpers, or by simply dragging and dropping the desired nodes. Schema identity constraints

Expanded Source Control System Support

Based on customer feedback, we’ve completely reworked the source control system interface in XMLSpy and also added the same level of source control support to UModel, our UML modeling tool, allowing both products to intelligently integrate with all major SCM tools. Once a project is bound to a version control system, XMLSpy automatically monitors the status of all files and prompts the you to check out a file whenever you starts to modify the document. In addition, the actual state of each file is shown through checkmarks or locks in the upper right corner of each file icon. What do you think of these new features? What would you like to see added to the next version of XMLSpy? Let us know by commenting below.

Tags: Altova, v2009, XML Editor, XML Schema, XMLSpy, xpath

Altova customer Recordare builds MusicXML-based solution

January 20, 2009/in /by Niki Devgood

Case Studies

Recordare® is a technology company focused on providing software and services to the musical community. Their flagship products, the Dolet® plugin family, are platform-independent plugins for popular music notation programs, facilitating the seamless exchange and interaction of sheet music data files by leveraging MusicXML. Dolet acts as a high quality translator between the MusicXML data format and other applications, enabling users to work with these files on any conceivable system, including industry leading notation and musical composition applications Finale® and Sibelius®. The list of MusicXML adopters also includes optical scanning utilities like SharpEye or capella-scan, music sequencers like Cubase, and beyond. Dolet increases the MusicXML support in all of these programs and promotes interoperability and the sharing of musical scores. In creating the Dolet plugins, Recordare used Altova’s XML editor, XMLSpy, for editing and testing the necessary MusicXML XML Schemas and DTDs, and the diff/merge tool, DiffDog, for regression testing.

The Challenge

Music interchange between applications had traditionally been executed using the MIDI (Musical Instrument Digital Interface) file format, a message transfer protocol that has its roots in electronic music. MIDI is not an ideal transfer format for printed music, because it does not take into account the multitude of notations (e.g., rests, repeats, dynamics, lyrics, slurs, tempo marks, etc.) that convey much of the meaning. MusicXML is an open, XML-based file format specifically created to encapsulate musical notation or digital sheet music data that was built on top of previous formats, MuseData and Humdrum. XML lends MusicXML the power and flexibility to be easily accessed, parsed, rendered, and otherwise manipulated by a wide variety of automated tools, and its general acceptance as a standard makes it an ideal format for scoring using computer technology. Since its original release by Recordare in January of 2004 (version 2.0 was released in June 2007), MusicXML has gained acceptance in the music notation industry with support in over 100 leading products, and is recognized as the de facto XML standard for music notation interchange. These products would not have adopted MusicXML unless it could be used to exchange data with industry-leading applications like Finale and Sibelius. By developing advanced plugins for popular music notation suites, Recordare would be able to deliver to their customers all of the advantages that XML can bring for data exchange and standardization.

The Solution

Below is an example showing the score of the first few measures of Beethoven’s An die ferne Geliebte, Op. 98 as it is written in sheet music: and a small snippet of the same piece translated to MusicXML: The MusicXML-based Dolet 4 plugins for Finale and Sibelius provide a more accurate and usable representation of sheet music than Standard MIDI translation. For example, the images below show the same piece of music. On the left is a Finale 2009 rendering of a MIDI file exported from Sibelius, and on the right is the same application’s interpretation of a MusicXML 2.0 file exported from the same version of Sibelius.
In the MIDI rendition, vital information like chord symbols, lyrics, slurs, articulations, and even title and composer are omitted from the translation. In addition to providing native support for MusicXML, the recently released Dolet 4 for Finale and Dolet 4 for Sibelius plugins enhance the capabilities of these programs by adding advanced features like:

Batch translation
More accurate and reliable data exchange
More formatting control
Support for the MusicXML XML Schema (in addition to the DTD)

In developing the plugins, Recordare was subject to specific requirements dictated by the Sibelius and Finale applications. The Sibelius plugin was programmed in ManuScript, and is one of the largest plugins ever written in that language. Finale, on the other hand, requires plugins to have a C++ core, and Recordare implemented this, adding MusicXML logic in Java and a JNI layer to provide the two-way Java/C++ communication. Recordare’s Dolet plugins are now critical aspects of the music preparation process for many television and film scores as well as new music publications. Errors in translation need to be fixed in maintenance updates, while ensuring that no new errors are introduced into these complex translation plugins. Regression testing of the MusicXML file produced by the Dolet plugins is thus an essential part of Recordare’s quality assurance process. Recordare used Altova’s DiffDog in the development of the Dolet plugins. XMLSpy was used to test and edit their DTDs and XML Schemas, and DiffDog for regression testing the MusicXML files produced by the software. Recordare has several regression test suites covering a wide range of musical repertoire, from baroque to hip-hop. DiffDog allows easy differencing of multiple runs of these test suites, including the ability to ignore differences in XML metadata elements such as software version and XML creation date that always change across test cases. Recordare has used Altova’s XMLSpy XML editor to edit the MusicXML DTDs and XML Schemas, starting with the use of XMLSpy 3.5 (released in 2001) to create the earliest alpha and beta versions of the MusicXML DTD. Version 2.0 of MusicXML added a compressed zip version of the format, similar to what is used in other XML applications like Open Office and Open XML. XMLSpy 2008 Enterprise Edition’s comprehensive support for zipped XML files made it easy to test this new feature together with the Dolet for Finale plugin.

A small portion of the extensive MusicXML schema shown in XMLSpy’s graphical XML schema editor

XMLSpy’s support for XQuery has also contributed to Recordare’s regression testing efforts. In response to a customer request, Recordare now exports XML processing instructions from the Dolet for Sibelius plugin when it encounters a musical feature that it is unable to translate correctly. A simple XQuery execution to search for all the processing instructions in the XML files in a given folder lets Recordare check for the presence of these restrictions within each test suite, and then compare the resulting XML files using DiffDog between runs of the test suite. Recently, customer demand led Recordare to develop an XSD version of the MusicXML format. XMLSpy Enterprise Edition was used to develop and test the schemas. Schema validation, schema restriction and extension, and automatically-generated schema documentation were all able to be tested using XMLSpy’s features.

The Results

The Dolet plugins are extensions for common industry software that harness the built-in capabilities of the MusicXML format to make musical scores truly interchangeable across disparate systems and toolsets. These plugins have the capacity to render accurate and meaningful musical notation based on the powerful MusicXML specification. The leading XML Schema editing capabilities in XMLSpy and the strong XML and directory differencing support in DiffDog enabled Recordare to write and polish the MusicXML schemas and perform regression testing on the Dolet plugins. The resulting high quality of the schemas and software has made MusicXML and the Dolet plugins a key element of the toolkit for composers, arrangers, publishers, copyists, and engravers throughout the industry wherever printed music is used. Try XMLSpy, DiffDog, and the other Altova MissionKit tools for yourself with a free 30-day trial.

Tags: Altova, Altova XMLSpy, Case Study, diff merge tool, DiffDog, MusicXML, Recordare, XML Editor, XML Schema, XMLSpy

Case Study: Equifax

August 21, 2008/2 Comments/in /by Niki Devgood

Check out the case study below to learn how leading US credit reporting entity Equifax® built an advanced SOAP interface for their identity verification and authentication Web service.

Overview

Equifax is a leading credit reporting entity and provider of analytical and decision support tools. Their real-time authentication system, eIDverifier, offers government and businesses personalized online security measures that help protect them against fraud and comply with federal legislation. The eIDverifier process is used within e-commerce and other online applications to authenticate users’ identities based on their answers to personalized questions drawn from Equifax’s extensive data stores. The authentication process consists of five steps:

Integrity Check – eIDverifier standardizes and screens applicant-provided information to test for data inconsistencies and irregularities.
Pattern Recognition – A pattern recognition algorithm is conducted on each transaction. For example, a velocity parameter determines the number of times an applicant has applied for authentication in a specific time frame.
Identity Validation – To confirm an identity’s legitimacy, eIDverifier uses a “waterfall” approach in gathering validation information from multiple data sources. This means that if the identity cannot be validated with the first data source, eIDverifier will proceed to the next data source until the identity is validated.
Interactive Query – eIDverifier presents multiple-choice questions to the applicant based upon “shared secret” information that should only be known to the applicant and Equifax. The question sets are customizable to meet individual risk thresholds.
Decision Logic / Output Assessment – There are two output components to eIDverifier – an assessment score and reason codes. The assessment score indicates the likelihood of an applicant presenting fraudulent information, while reason codes provide important details on questionable information and highlight any discrepancies between the consumer’s application information and Equifax data sources.

eIDverifier relies on the SOAP protocol to send messages defining these interactions back and forth between the client interface and the Equifax servers. Third party institutions license the eIDverifier SOAP interface for use within their online application processes, enabling them to integrate its functionality and access information contained in Equifax’s databases.Equifax uses the XMLSpy XML Schema editor to graphically design the XSDs that serve as the foundation for their SOAP interface.

The Challenge

Equifax needed a sophisticated tool for designing the XML Schemas that would define the data types for their Web service, as well as a mechanism for creating the WSDL documents that would describe the interface as a whole. As a Java shop, Equifax needed a solution that would be compatible with their other development tools, and that would work seamlessly with the Eclipse IDE. Though there are plenty of Java tools available that have the capacity for XML Schema development, XMLSpy presented the most attractive option for schema design because of its comprehensive graphical design and editing options.The Equifax development team took a further step to simplify their Web services creation, using XML Beans and the Codehaus XFire/CXF Java SOAP framework to auto-generate WSDL from their XML Schemas.

The Solution

eIDverifier relies on a variety of different technologies to bring identity verification and authentication to its clients. XMLSpy provides the following benefits:XML Schema

XML Schema is used to express the structure of the data, as well as the individual elements and attributes that it is comprised of. Because a large portion of the data relies on end-user input in the form of address, phone number, driver’s license number, etc., it is vital that this information is in a format that can be digested by the system.Using XMLSpy’s graphical XML Schema editor, the Equifax development team was able to easily visualize and maintain the structure of their XML Schema. A portion of the schema that was created appears below:

SOAP interface

This data type definition provides the syntax, and dictates the structure, for the data that is transmitted by the eIDverifier Web service.

XMLSpy’s unique graphical XML Schema editor allowed the Equifax development team to create and maintain a complex schema definition without writing any code manually. They were also able to automatically generate human-readable documentation that can be used to present the architecture for review at any time in the development process, and that describes each element and attribute in detail.

WSDL

The processes executed by eIDverifier are described by a WSDL document that incorporates the XML Schema to provide information about data types, functions, and other interface details to the client – defining and dictating the actions taken by the client application to send and retrieve information between the end-user and the Equifax servers. The Equifax team chose to autogenerate a WSDL document using the Codehaus XFire/CXF framework. The XML Schema was used as the basis for an XMLBeans implementation, which was then compiled as a Java service class. Once the eIDverifier service was exposed, XFire automatically generated a WSDL – the WSDL is shown below in the XMLSpy graphical WSDL editor.

This WSDL serves as the basis for the eIDverifier application, defining the ports and messages that make up the communication infrastructure of the Web service.

The Results

The eIDverifier SOAP interface allows external applications to access Equifax’s backend data stores, exposing it as a Web service and enabling them to retrieve secure information without jeopardizing the integrity of the Equifax mainframe. Utilizing WSDL and SOAP, and surrounded by Java architecture, eIDverifier is able to confirm user identity by returning a set of multiple choice questions based on the secure data maintained by Equifax. SOAP interface XMLSpy enabled the Equifax team to quickly and easily create a graphical schema representation and the matching documentation to serve as the basis for the Web service. It also allowed the development team to focus on their Java code, rather than the intricacies of XML Schema and WSDL design. The Altova MissionKit provides numerous tools for advanced Web services development, from the graphical XML Schema and WSDL editing discussed here, to SOAP debugging, and even graphical Web services generation and data mapping. Download a free trial to check it out for yourself.

Tags: MissionKit, SOAP, Web services, WSDL, XML Schema, XMLSpy