Customizing a FlowForce Server Job


In our earlier post titled Automate Data Mapping and Transformation with FlowForce Server, we created a job called SimpleMapAndTransform to automate data mapping with MapForce Server and creation of html reports by StyleVision Server. After the FlowForce Server job ran several times, we have accumulated many output files in the same folder we use to process input files, as well as temporary intermediate files in the workFiles folder, as seen in the image below.

FlowForce Server job execution log and working folders

In this post we will enhance the job to create more orderly results and remove unneeded temporary files.
Read more…

Tags: , , , ,

Automate Data Mapping and Transformation with FlowForce Server


Altova FlowForce Server, launched on April 29, 2013, includes numerous enhancements over earlier beta releases and one of the most exciting new features is the implementation of StyleVision Server. Now a FlowForce Server job can automate a complete data transformation workflow by executing MapForce Server for data mapping and pipelining results to StyleVision Server to render a variety of output formats.This post describes a straightforward FlowForce Server mapping and transformation job as illustrated in these messages from the FlowForce Server Log, with the most recent step at the top of the list:

FlowForce Server Job Log showing MapForce Server and StyleVision Server job steps

We’ll start with the GPS log files created by a digital camera. We wrote about these files last January in the post titled Process Multiple Input Files in a Single Data Mapping. We’ll use the mapping output with a StyleVision SPS stylesheet adapted from the XPath Enhances XML Reports post to produce a time and elevation report for each file.A FlowForce Server For-each job step repeats based on the result of an expression. We can use For-each to build a list of files in a folder, then repeat one or more steps for each file. Here is how it looks in the job configuration page:

FlowForce Server data mapping job step

The line labeled Execute function defines the mapping to be used by MapForce Server, and the input parameter {file} refers to each file in the list C:CameraGPSexample*.LOG.We can add an execution step to instruct StyleVision Server to perform the transformation:

FlowForce Server transformation job step

The data mapping creates output files by adding .gpx to the name of the input file, and now we can define the transformation input using the {file} variable with the new file suffix. We chose to create .html output, but we could just as easily create other formats for a multi-channel publishing implementation.The transformation working directory is the location where StyleVision Server unpacks the contents of the .pxf file containing the stylesheet, XML Schema, and other needed components. Using a dedicated working folder will keep the workflow more organized.We want to allow network users to drop new .LOG files into the C:CameraGPSexample folder and we want run the FlowForce Server job on a regular schedule, but we don’t want to process the same files over and over. We can define one more job step to move the processed file to a different location:

FlowForce Server file move job step

The complete FlowForce Server job is a series of three steps that loops for each .LOG file found in the folder. We can set up a repeating trigger for the workweek or any other appropriate schedule:

FlowForce Server calendar-based job trigger

Here is a portion of a .LOG file created by the camera that is an example of one input file:

.csv input file example

We can drop this file into the C:CameraGPSexample folder, where it will be processed based on the FlowForce Server job trigger:

Workflow folder structure

When the timer triggers execution of the FlowForce Server job, the Web interface Job Log page displays this series of messages for the complete job:

FlowForce Server job log for complete job execution

The contents of the C:CameraGPSexample folder now look like this:

Completed work files after FlowForce Server job execution

We can examine the 121130.LOG.gpx file in XMLSpy:

XML file created by MapForce Server job step

And we can open the .html file in any Web browser:

.html output from StyleVision Server job step

In future blog posts we will enhance this FlowForce Server example to illustrate jobs with error handling and more complete cleanup of working files.FlowForce Server is available for Windows, Linux, and soon for Mac OS platforms. To get started yourself, click here to download a free trial!

Tags: , , , , ,

New Server Products for Data Transformation


Altova Server ProductsOn the heels of our RaptorXML announcement earlier this month, we are very excited to tell you about the availability of three more new products that round out the Altova server software product family.

These high-performance server products are optimized for multi-CPU servers and are highly complementary when used together for defining and managing information pipelines. Cross-platform support is immediately available for Windows and Linux, with MacOS support expected just around the corner in May. Supported languages include English, German, Spanish, and Japanese.

Let’s take a look at each of these new products in detail.

FlowForce ServerAltova FlowForce Server

After receiving valuable customer feedback during the beta period, we have announced general availability of FlowForce Server, a powerful new tool for managing multi-step, enterprise-level data aggregation, processing, and reporting tasks. The straight-forward FlowForce Web browser interface makes it easy to configure and monitor jobs, whether they require a single step, or multiple cascading steps starting with data aggregation and conversion, and completing with multi-channel report generation.

FlowForce Server Job Log

A flexible and highly customizable system of job triggers lets you schedule FlowForce Server job execution based on each job’s unique requirements. Jobs can be triggered by date and time, by the arrival of files in hot folders, or by http command – and you can define multiple triggers per job. To take full advantage of server resources and meet the demands of busy data transformation workflows, multiple jobs – even multiple instances of the same job – can run simultaneously on FlowForce Server.

FlowForce Server Job Triggers

FlowForce Server provides strong security measures through user, role, job credential, and permission definitions that enable multiple simultaneous jobs while preventing unauthorized access to restricted data repositories or system resources.

So, what kind of jobs can you set up on FlowForce Server? Most customers will want to use it in conjunction with MapForce Server and/or StyleVision Server for high-performance automation of data integration and multichannel publishing projects that doesn’t require writing or compiling any source code.

FlowForce Server enterprise workflow

MapForce ServerAltova MapForce Server

You may already be familiar with MapForce, Altova’s desktop developer software for aggregating, mapping, and converting data. We describe it as an any-to-any data mapping tool because it lets you move between any combination of XML, SQL database, flat file, EDI, Excel, XBRL, and Web services data. With the introduction of MapForce Server, it’s now easy to fully automate those transformations that have been defined in MapForce.

MapForce Server performs data transformations based on preprocessed and optimized data mappings stored in MapForce Server Execution files prepared by MapForce and uploaded over a network. Preprocessing enables faster performance and reduced memory footprint for most data mappings.

When operating under the management of FlowForce Server, MapForce Server is a powerful transformation engine suitable for the high-volume, quick turn-around demands of today’s leading edge data centers. The standalone version is suitable in a department level or other small environment that does not require the extensive multi-tasking, job scheduling, and security features of FlowForce Server.

StyleVision ServerAltova StyleVision Server

StyleVision Server provides high-performance automation of report and document generation tasks based on templates designed in Altova StyleVision. A graphical XSLT stylesheet and report design tool, StyleVision helps you create one design to render XML and XBRL data in Web and print formats including HTML, Word, RTF, and PDF, simultaneously. Using StyleVision Server, you can generate reports from XML and XBRL data on a recurring basis faster than ever before.

Like MapForce Server, StyleVision Server is an important option for document production by FlowForce Server, and the standalone version can be run from the command line and is suitable in smaller deployments that may not require all the features of FlowForce Server.

This new line of Altova server products represents a huge leap forward in effective automation of essential business processes, whether they require a single step, or multiple cascading steps starting with data aggregation and conversion via MapForce Server and completing with multi-channel report generation via StyleVision Server. These high performance workflow servers help customers design information pipelines quickly and easily, and then deploy them in a flexible way, utilizing the power and speed afforded by multi CPU servers. We’re excited to offer this new family of server products to our customers developing data integration and reporting projects using Altova MissionKit developer tools, as well as welcoming new customers who need a multi-platform, lightning fast orchestration engine.

Download a free fully-functional 30-day trial!

As with all Altova tools, we invite you to try these new server products before making a purchasing decision. Download a free, 30-day trial now!

Tags: , ,

Announcing RaptorXML, the Hyper-fast XML and XBRL Server


  clip_image001We are very excited to be at the XBRL 26 Conference in Dublin, Ireland today to announce a brand new server product in the Altova family of XML and XBRL tools! Altova RaptorXML is a hyper-fast XML and XBRL validation and processing server. It’s Altova’s third-generation XML and XBRL engine, built completely from scratch to help organizations efficiently validate, process, transform, and query the vast and ever-increasing amounts XML and XBRL data being generated as a result of XBRL compliance regulations and myriad other big data trends. RaptorXML is written to be highly scalable for today’s multi-CPU and multi- core computers and servers. This, along with high performance code optimizations and an extremely low memory footprint, has helped make RaptorXML a lightning-fast XML and XBRL server that can meet the demands of today’s data processing applications. Simply put: we architected RaptorXML to combine the performance benefits afforded by modern parallel computing environments with strict compliance to the latest versions of all relevant XML and XBRL standards. RaptorXML includes support for the very latest versions of all relevant standards and has been submitted to rigorous regression and conformance testing. The server will be available in two versions, both of which are available for Windows, Linux, and MacOS platforms. RaptorXML Server supports validation and processing of:

  • XML 1.0 & 1.1
  • XInclude 1.0
  • Xlink 1.0
  • XML Schema 1.0 & 1.1
  • XPath 1.0, 2.0 & 3.0
  • XSLT 1.0, 2.0 & 3.0 (subset)
  • XQuery 1.0 & 3.0
  • And more

RaptorXML+XBRL Server supports all the features of RaptorXML Server, with the addition of processing and validating the XBRL family of standards:

  • XBRL 2.1
  • XBRL Dimensions
  • XBRL Formula 1.0
  • XBRL Functions
  • XBRL Definition Links

Developers creating solutions using Altova MissionKit XML development and XBRL development tools will be able to power their server applications with RaptorXML for hyper-performance, increased throughput, and efficient memory utilization, giving them the opportunity to validate and process large amounts of XML or XBRL data cost-effectively.  Check out the complete list of supported XML and XBRL standards and more details on this groundbreaking new server product. RaptorXML will be available to download and purchase in May. clip_image003

Tags: , , , , , ,

Editing, Converting, and Generating JSON


As the use of JSON as a data transport protocol increases, I thought it would be useful to take a look at JSON support in XMLSpy. There’s been much debate about advantages of JSON vs. XML, but when you boil it down, there are simply some cases for which JSON is the best choice, and others where XML makes more sense. This article on the XML Aficionado blog unpacks this topic quite well.
While you might need to choose between JSON and XML depending on the development task at hand, you don’t have to choose between code editors – XMLSpy supports both technologies and will even convert between the two. Let’s take a look at how that works.
Editing JSON
To make JSON editing as easy as possible, Altova extended its intelligent XML editing features to the JSON editor. If you choose to edit JSON in text view, XMLSpy helps you along with syntax coloring, bracket matching, source folding, entry helper windows and menus, and so on.

clip_image001

Here’s that same JSON file in grid view, which offers a graphical representation of the JSON structure with drag and drop editing. Both views provide JSON syntax checking and advanced error checking features.

clip_image002

Another common requirement is converting XML to/from JSON, which is a one-click option on the XMLSpy convert menu. The JSON data above has been converted to valid XML:

clip_image003

Having JSON editing and conversion functionality directly inside the XML editor is quite useful, especially if you’re using the free Eclipse or Visual Studio integration package.
Check it out and let us know what you think.

Tags: , ,

Web Interface Simplifies Automation of Data Transformations


FlowForce Server is a new product designed to provide automation of data transformations performed by dedicated high-speed servers. FlowForce Server can start jobs based on a variety of triggers, runs multiple jobs simultaneously, and can even run multiple instances of the same job, depending on workflow.Monitoring all these complex activities is critical to success in a busy production environment. The FlowForce Server Web interface includes customizable views into operations, simplifying management from anywhere on the network.

FlowForce Server viewed in a Web browser window

The screen shot above shows the Home page of the FlowForce Server Web browser interface, displaying all currently running jobs and active triggers. Six instances of the gpxElevationUSGS job are running, each identified by a unique job ID. Four job triggers are also active, three watching hot folders, and one based on a timer. Each FlowForce Server job automates a MapForce data transformation.The orange arrows below each grid are clickable update buttons, and the job names link to the definition pages for each job.The blue headings at the top are also clickable buttons. The Log link displays the Log View, a detailed history of all system activity, shown in a truncated version below.

FlowForce Server Log View Shows History of Operations

Each message line describes one step in one instance of a FlowForce Server job, and may have links to more information, as we described in our earlier blog post titled Automate Data Transformation with FlowForce Server. If the status is anything other than 0, the more link opens a detailed error message.

Description of a single FlowForce Server job step

In a busy environment, all this can be just too much information. In that case, the Log View offers several alternatives to help you find critical information quickly. The options bar above the list lets you filter items displayed by a particular job or Message severity.

Filter FlowForce Server Log messages by date, by job, or by severity

If you only want to know if anything unexpected occurred, select Warning as shown above to hide all normal Info messages.You can also sort the list by any of the column headings Date, Severity, Module, User, or Instance ID in ascending or descending order. When the FlowForce Server is running multiple simultaneous jobs and instances, it’s very likely individual steps from different jobs will be shuffled. Sorting by Instance ID can let you more easily review the job history.Click here to learn more about FlowForce Server Beta 3 at the Altova Web site, or visit the FlowForce Server Beta 3 download page to get started automating data transformations in your data center!

Tags: , ,

Compare Folders Fast and Efficiently with DiffDog


Three user preferences in DiffDog work together to accelerate directory differencing tasks into hyperdrive, especially when you compare folders with thousands of files.DiffDog can perform specialized comparisons. For instance, XML comparison mode commands DiffDog to determine whether two files contain equivalent XML content, even if they would be different in a text comparison because of spacing, line-ending, or attribute order variations. For the fastest directory comparisons, choose Quick Comparison mode, which only looks at file sizes and modification dates.

DiffDog Quick Comparison toolbar button

When you compare large directories, most often you want to quickly locate differences. Choose the Hide Equal setting to avoid listing all identical files.

DiffDog Show/Hide equal toolbar button

Show/Hide empty directories should be set to Hide, otherwise subdirectories that contain identical files will be listed, but since their contents are equal, none of the files are shown.

DiffDog Show/Hide empty directories toolbar button

After you choose settings and click the Compare button, DiffDog flies through the directory comparison.

DiffDog Quick Comparison progress bar

When the comparison is finished DiffDog displays a concise listing showing only the differences, with a summary message at the bottom of the window. The toolbar in the screenshot below illustrates the settings for Quick Comparison on, Show Equal files off, and Show Empty directories off.

DiffDog directory comparison results

DiffDog saves your preferences to apply to the next folder comparison, and DiffDog even includes a quick synchronization tool to merge folders in either direction.

DiffDog Synchronize directories summary

Find out for yourself how fast DiffDog can run your folder comparisons – download a fully-functional, free 30-day trial!

Tags: , , ,

Automate Data Transformation with FlowForce Server


Altova designed FlowForce Server to provide comprehensive automation, management, and control over data transformations performed by dedicated high-speed servers. FlowForce Server Beta 3 is currently available to users of MapForce Enterprise and Professional Editions at no charge during the beta test period.FlowForce Server can provide hot folder automation of data mappings and maintains a detailed activity log users can monitor remotely in a Web browser window. The screenshot below shows the log for FlowForce Server running the MapForce data mapping CameraLogToGPX we wrote about in the blog post titled Process Multiple Input Files in a Single Data Mapping. This mapping used wildcards to specify multiple input files for processing.
FlowForce Server job log

It only takes a few minutes to set up, run, and review the results of jobs like this on FlowForce Server. Wildcards or Hot Folders?Wildcards and hot folders increase the complexity of a data transformation workflow, and using them successfully requires careful planning. Let’s take a minute to look a little deeper at the scenario we want to implement.Assume we are the IT department in a company that publishes nature and hiking guides. We employ photographers who go out trekking and record their routes as they go, using the GPS tracking feature of their digital cameras. We want to convert the camera GPS log files to XML-based .gpx format for mapping and other processing.We will publish a folder on our network where photographers can drop off their GPS log files. This will be the hot folder FlowForce Server watches for new files to supply as input to the CameraLogToGPX mapping. We only need to process each input file once. So, after the data transformation is complete, we can remove the input file from the hot folder. We also want to place the output file in a separate folder. This suggests the following FlowForce Server job steps:

  • Look in the hot folder to see if new a input files has arrived
  • Perform the data mapping on the input file and place the output file in a separate folder
  • Move the input file to a permanent location

The diagram below shows a folder structure we can use for the workflow, with files ready to drop into the hot folder for processing:

Hot folder structure

The hot folder is C:CameraGPShotFolder and the generated .gpx files will be placed in C:CameraGPSoutputFiles. When the data mapping is done, input files will be moved to C:CameraGPScompletedInput. Deploy the Mapping on a ServerMapForce Beta is a component of FlowForce Server that works just like MapForce, and adds a feature to deploy mapping files to a FlowForce Server. Existing data mappings need little or no special preparation for deployment.The only thing we need to think about are the filenames of the input and output files. We will instruct FlowForce Server to provide the input filename as a job parameter as new files arrive in the hot folder for processing. The original mapping assigned a wildcard filename for the input component, which is no longer needed. We will also want FlowForce to specify the location of the output file.We can open the mapping in MapForce Beta, remove the filename from the input component, and add a remove-folder filepath function to the output file so FlowForce Server can set the destination. The screenshot below shows the new filename definitions in the mapping.

Mapping file names

In the MapForce Beta dialog to deploy the mapping, we can choose to immediately open the mapping in a FlowForce Server job definition window in a Web browser to finish defining the job operations.

Deploy mapping dialog

Defining the Job in FlowForce ServerThe screenshot below shows the complete job steps defined in a FlowForce Server job properties window:

FlowForce Server job definition

The job trigger is defined at the bottom of the window. Every 30 seconds, FlowForce Server will check the hot folder. If the contents have changed, FlowForce Server will execute the job steps. Each Execution step could be a deployed MapForce mapping, a system step, or even another FlowForce Server job.The name of each new file entering the hot folder becomes the parameter called {triggerfile} that we use in the mapping step as the input filename, and in the move step as the name of the file to be moved.The Working-directory parameter in the mapping step defines where the output files will be placed.FlowForce Server also includes features to set automatic run and stop times for jobs, user permissions and roles, and Queue settings to define minimum time between job runs and maximum parallel instances of a job.In our scenario, we are likely to receive multiple input files in groups as they are copied from photographers’ memory cards. Multiple parallel runs can greatly improve throughput. As a rule of thumb, you might want to match the number of cores or CPUs in the machine running FlowForce Server.

FlowForce Server job queue settings

Exploring the Job Log

Each time FlowForce Server runs our job, six lines are added to the Log View shown in the illustration at the top of this post. The first and last lines record the start and completion of the job, and each Execution step generated its own start and completion messages. The phrase “completed with status: 0” means the step was successful with no errors.

FlowForce job log

We can click the more links for a detailed report on each Execution step. The screenshot below shows the message for the MapForce mapping Execution step:

FlowForce job detail

Each file dropped into the hot folder generates an individual FlowForce Server job instance, even if multiple files are added as a group. This makes it easy to track any individual input file that generates an error.When we dropped four files into the hot folder, FlowForce Server ran four jobs, and the contents of the output folder looked like this:

Job results folder

Click here to read more about FlowForce Server Beta 3 at the Altova Web site, or visit the FlowForce Server Beta 3 download page to get started automating data transformations in your data center!

Tags: , ,

Process Multiple Input Files in a Single Data Mapping


Typical data conversion tasks require processing numerous input files that arrive in batches. Altova MapForce includes features that let you handle groups of files with minimal intervention.For instance, we recently copied a set of files from the memory card of a digital camera with GPS support. Each .LOG file is a CSV containing GPS coordinates for a single route.

Directory list of source files for data conversion

We quickly designed a mapping to convert the CSV data to XML-based .gpx format and processed all three files to generate three output files in a single execution:

List of automatically generated output files

Read more…

Tags: ,

Expect the Unexpected – Altova MissionKit Solves a Number Format Mystery


Every time you receive data from an outside source, there is a chance it won’t arrive in the form you expect. This can require special accommodations for the rare and unlikely to make a real-world data mapping and transformation solution robust and reliable.

We processed literally dozens of .gpx files, containing hundreds of coordinates each, through the MapForce mapping we wrote about in the blog post Web Service as a Look-Up Table to Refine GPS Data. Then one day we ran a new file and encountered the error below, which caused the mapping to fail:

Error message during mapping of Web services data

Reaching into the Altova MissionKit to combine features of MapForce and XMLSpy, we quickly diagnosed the issue and developed a solution we can also reuse in future mapping projects.

Read more…

Tags: , , ,

Brand New Altova Web Site


We are so excited to announce the launch of our new, improved, completely redesigned Web site! The new Altova.com is cleaner, simpler, and more beautiful than ever.

clip_image002

Our goal was to make our site easier to navigate so that you can find exactly what you’re looking for quickly, whether that’s Product information, Support resources, or technical reference materials. Of course, it’s as simple as ever to download a free trial or update your software to the latest version.

clip_image004

At the same time, we hope the new site will help you discover a little something you didn’t know about Altova. For instance, did you know that we have a comprehensive line of Database Tools in addition to the XML Tools and Developer Tools that we’re known for? Did you know that we offer free, online product and technology training? And have you tried our new, cloud-based project collaboration and team management tool?

clip_image005You can learn all about how Altova products and supported technologies can help your team meet critical business needs in our completely revamped Technology Solutions Center.

Despite all the planning, proposals, testing, and reworking that go in to a new Web site, the true test of success is how the site works for you, our customers. Please share any feedback or suggestions you may have for the new site, either by leaving a comment here on the blog or contacting our Marketing team by email. We look forward to and greatly appreciate your input!

Tags: , ,

Version 2013 Service Pack 1 Delivers Windows 8 Support and More


clip_image002 On November 15, 2012, we released Service Pack 1 of Altova MissionKit and the complete line of Altova XML, SQL, and UML tools. This service pack delivers the usual round of bug fixes as well as some important new updates to existing support:

  • Windows 8 support. Altova’s desktop applications now run under Windows 8 in addition to Windows 7 and earlier platforms.
  • Windows Server 2012 support. Altova tools now run on Windows 2012 Server in addition to earlier versions.
  • Visual Studio 2012 integration. Altova tools that support integration with Visual Studio have been updated to support the latest version of Microsoft’s IDE.
  • Internet Explorer 10 support. Altova desktop products with browser integration now all support Internet Explorer 10. In addition, the Authentic Browser plug-in supports IE 10 in desktop mode.
  • XMLSpy 2013 Chinese language version. Version 2013 of XMLSpy is now available in Chinese as well as English, German, Japanese, and Spanish.

Version 2013 Service Pack 1 is a free update for all customers of Version 2013 products, as well as any customer with valid Support and Maintenance. Download and install SP1 now.

Tags: ,

The Altova MissionKit 2013 Introduces Seamless Integration of XMLSpy, MapForce, and StyleVision in Java Applications


The Altova MissionKit 2013, empowers developers to integrate XMLSpy, MapForce, or StyleVision functionality seamlessly in custom Java applications for Windows. This frequently-requested capability adds to existing support for integrating these MissionKit tools in Visual Basic or C# applications, giving developers flexibility to add some or all XMLSpy, MapForce, or StyleVision views and functionality to their own custom apps.

Version 2013 of XMLSpy, MapForce, and StyleVision include a new API that allows each to run inside a window within a Java application developed using the Java Abstract Window Toolkit (AWT) or Java Swing. This functionality allows MapForce 2013, for instance, to be embedded in larger applications where data mapping and transformation is only one requirement.

MapForce running in a Java ActiveX window

Altova provides sample applications with XMLSpy, MapForce, and StyleVision illustrating use of the Java API. You can access the sample applications from the command line or from within Eclipse.

Read more…

Tags: , , , , , , ,

Trick or Treat with Altova at DevConnections!


devconnectionsIt’s no trick – we will be in Las Vegas October 30 – November 1 at the Bellagio Hotel for the fall DevConnections event, in booth #408. If you are attending the event we would love for you to stop by our booth, where we will be showcasing MissionKit, Altova’s suite of XML, SQL and UML tools.

Altova MissionKit 2013 provides specialized functionality for Microsoft developers and IT pros, including seamless integration with Visual Studio®, C# and C++ code genjack-o-lanterneration, powerful functionality for SQL Server® database management and reporting, integration with SharePoint® Server, functionality for working with Open XML (OOXML), diff/merge for Word documents, and much more.

While you are at our booth, don’t forget to enter our drawing for a chance to win a free Altova MapForce Basic Edition license. If you also mention that you saw this blog post – we might have a special treat for you too!

Tags: ,

Do You Believe in (XML) Magic?


One of the most exciting features in our recent Version 2013 release is Smart Fix XML validation in XMLSpy. Smart Fix is so magical it can revolutionize XML validation as we know it. While the XMLSpy validator has always been powerful, with new features added often, Smart Fix really delivers the “wow” factor: automatic error correction.

clip_image002 Let’s take a look at how it works.

Read more…

Tags: , , , ,

Using Table Row and Column Conditions in StyleVision 2013


The ability to conditionally hide table rows or columns has been a popular feature request from StyleVision users. StyleVision 2013 adds the capability in the form of conditional expressions on table rows and columns. Conditional processing can be set on individual columns and rows of static and dynamic tables, as well as on column and row headers, to display or hide the column, row, or header depending on the truth of the condition. The conditions are specified using StyleVision’s XPath Expression editor. If a condition evaluates to true, the column, row, or header is displayed; otherwise it is not (i.e., is hidden).

StyleVision 2013 Edit XPath Expression dialog

Let’s take a look at some scenarios for which this new feature is exceptionally useful.

Read more…

Tags: , ,

Watermarks in StyleVision 2013


Watermark support has been added in StyleVision 2013, in response to customer requests. A watermark is text or an image that is displayed on the background of each page of a document section. Watermarks are often used in draft and confidential documents in order to emphasize a document’s status or security constraints.

In StyleVision 2013, watermarks may be used for print-oriented output formats, including PDF, RTF, and Word (Open XML). In keeping with StyleVision’s overall design model, with a single layout design used for multiple output types, you only need to define watermarks once, and they are applied to all print-output views.

Here’s an example of a StyleVision design containing a watermark:

StyleVision 2013 Word watermark example

Let’s take a look at the steps involved in using watermarks in StyleVision 2013 by modifying the QuickStart example (quickstart.sps) included in the Tutorials folder (installed with StyleVision), in order to produce the design shown in the image above.
In the Design view, if you select “Edit Properties” in the Initial Document Section, you’ll see a new “Watermark…” button. Clicking the button presents the Edit Watermark dialog:

StyleVision 2013 edit image watermark dialog

StyleVision supports both image and text watermarks. For both types of watermarks, the general idea is to specify:
· A condition used to determine when the watermark should be included (which can simply be “true()” in order to include the watermark on all pages)
· The watermark content (image or text, with related attribute setting preferences)
· Watermark size and position settings

For our example scenario, let’s assume we want to place a red “CONFIDENTIAL” text watermark across printed pages. To do so, we click the Text tab in the Edit Watermark dialog and:
· Enter “true()” for the condition
· Enter “CONFIDENTIAL” for the watermark text
· Select red for the watermark color

StyleVision 2013 edit text watermark dialog

StyleVision’s support for watermarks is powerful and flexible. You can include both text and image watermarks, for example, with different XPath conditional expressions used to specify what should appear when. You can also specify different watermarks (or pairs of image and text watermarks) for each document section in a StyleVision design.

Check out watermark options and the other new features in StyleVision 2013 by downloading a free 30-day trial version today!

Tags: , ,

Visit us at Oracle OpenWorld next week!


clip_image002
Fall is here, which must mean we are off to San Francisco, CA for the 2012 Oracle OpenWorld show.  If you are planning on attending come by and see Altova at the Moscone center from Sept. 30th – October 4th in booth #3408 for a demo of Altova’s tools for Oracle Users and see the amazing new features in the newly released Altova MissionKit 2013! clip_image004

While you are visiting our booth tell us about your latest projects and challenges, collaborate on best practices or let us show you some of the latest and greatest features available in MissionKit 2013, like Smart Fix XML Validation in XMLSpy, mapping SQL stored procedures in MapForce, watermark support in StyleVision, or our seamless integration options in Java applications.  Don’t forget to enter our drawing for a chance to win a free Altova MapForce Basic Edition license!

Tags: ,

Stored Procedures in Database Mappings


In response to user requests, the new release of MapForce 2013 includes robust support for stored procedures in database mappings as:

  • input components (for stored procedures that provide results)
  • output components (for stored procedures that insert or update data in a database)
  • or, stored procedures may be inserted as a function-like call (allowing users to provide input data, execute the stored procedure, and read/map the output data to other components)

The screenshot below shows the mapping of a stored procedure in SQL Server to create an XML file.

A MapForce database mapping with a stored procedure

Read more…

Tags: , ,

Altova MissionKit 2013 with Smart Fix


clip_image002

Altova Software Version 2013 was released today, and, as always, numerous features in this year’s autumn launch have been developed in response to customer requests. You asked for the ability to use SQL stored procedures in MapForce data mapping projects, to embed XSDs in WSDL documents in XMLSpy, to add watermarks in StyleVision, and to integrate MissionKit functionality flexibly in Java applications. All those features and more are there in Version 2013. We’ve also added some features that can only be described as magical. Let’s take a look at the highlights…

Read more…

Have you ever wanted a schema for Apache Ant?


Recently I was having some issues with the Ant files for a software project I was working on. I was editing one of them in XMLSpy but wasn’t getting entry help or code completion because I didn’t have a schema assigned to my script.

That seemed like a simple problem, so off I went looking for the schema.

I quickly found a Stack Overflow question looking for the same thing. The answer turns out to be, there is no Ant schema or DTD. The reasoning is straightforward. Above the challenge of making a flexible and comprehensive model of the Ant core, it would be able to include all the external tasks.

I immediately remembered that I’d been here before a couple years back, but this time I figured I’d use XMLSpy’s schema generation feature to try to do something about it…

 

XMLSpy XML editor

Read more…

Tags: , , ,

FlowForce Server Beta 3 is Now Available


Altova FlowForce Server Beta 3

MapForce 2012 users who need to automate data transformations in a centralized environment are enthusiastic about FlowForce Server, the new server product from Altova that automates execution of MapForce data transformations. Now we are pleased to announce the release of FlowForce Server Beta 3, offered as a free public beta test to all licensed users of Altova MapForce 2012 Enterprise Edition and MapForce 2012 Professional Edition.

The beta test period for FlowForce Server Beta 3 is extended until March 31, 2013.

FlowForce Server Beta 3 adds support for remote job requests via an HTTP client and job parameters that can be passed to any step in a job. When used together with the request interface, job parameters empower the HTTP client to specify input values in the job request.

FlowForce Server Beta 3 also permits any job to be called as a step within another job, implements individual job queues that make it possible to control server resources used by jobs, and adds many more refinements and enhancements.

FlowForce Server Beta 3 is available in a 32-bit version and a 64-bit version. The 64-bit version allows the MapForce Beta 3 module to process extremely large data files and provides compatibility with 64-bit database drivers.

Click here to read more about FlowForce Server Beta 3 at the Altova Web site, or visit the FlowForce Server Beta 3 download page to get started automating data transformations in your data center!

Tags: , ,

XML Summer Fun


Aloha! Summer is the time to hang loose, get some sun, and recharge your XML knowledge. Altova offers free, self-paced, Online Training in XMLSpy, MapForce, and StyleVision. We also have technology specific training in XBRL, XSLT, and HL7.

Don’t be a poser when your bro’s talk up XSLT transformation. Our classes will let you brag about some cherry surfing through gnarly XPath code.

XML getting you bummed? Brush up on XMLSpy techniques to ride a fat wave to a juicy completed project. Let MapForce put your data in the best form for your project and make your output major with StyleVision.

Don’t get cranked by the tax code. Use our XBRL training to become the Big Kahuna and shoot the pipe to the Surf Shack of Standards Compliance.
clip_image002
Grab your board, and surf over to Altova Online Training to catch some sweet knowledge.
You’ll say “Mahalo, Altova!”

Tags:

Web Service as a Look-Up Table to Refine GPS Data


Elevation data recorded by GPS devices is notoriously inaccurate, especially in hilly terrain like the Russian River Valley example from our earlier post.

The final elevation track plotted from the Russian River Valley GPX file is suspicious for several reasons. First, the graph shows we descended almost 50 feet below sea level. That’s hard to believe, since we were travelling along the bank of the river, only about 10 miles from the Pacific Ocean.

Altova StyleVision ChartSecondly, we were headed mostly west, following the river downstream, but the track shows a predominantly uphill trend.

We can evaluate the recorded GPS elevation data by comparing it to information available from the United States Geological Survey (USGS). The USGS operates a Web service that accepts latitude and longitude coordinates, and returns elevation data measured by NASA and assessed for accuracy based on over 13,000 control points in the continental United States.

Using the elevation Web service in an Altova MapForce mapping will let us extract each point from the GPX file, send the coordinates to the USGS Web service, and build a new GPX file with corrected elevation data.

Read more…

Tags: , , , , , , , , ,

We will be at TechEd! Will you?


clip_image001Altova will be exhibiting in sunny Orlando, FL from June 11 – 14 at Microsoft TechEd in booth #2338.  Come on down to the Orange County Convention Center and talk with our experts about the new features in the Altova MissionKit.

clip_image002
If you are attending TechEd, stop by our booth to see the latest features in the Altova MissionKit 2012r2, which include EPUB editing and validation in XMLSpy, sorting of data mapping results in MapForce, generation of code from UML sequence diagrams in UModel, RichEdit functionality in StyleVision and Authentic and much more! We hope to see you next week.

Tags: ,

New Case Study: Automating XBRL Data Collection and Processing


Case studies are a great way to see how other organizations use Altova® technologies to develop unique projects that meet their business goals. We’re often asked, however, what comes next. Did the project take off? Has it grown since the case study was published?

We’re happy to bring you a follow up to a case study we published last year about the not-for-profit Maryland Association of Certified Public Accountants (MACPA).
In the original case study, MACPA converted General Ledger and other financial data from siloed systems into XBRL – in house and on a budget – with the Altova MissionKit®. At that point, they were mapping the XBRL file to multiple external targets, including an Excel spreadsheet used to feed their Key Performance Indicators (KPI) system. (You can read the original XBRL case study here.)

Since that time, they’ve built on the foundation they created and developed a business intelligence dashboard driven by the XBRL files they generate in MapForce®. They used Altova FlowForce®, an application designed to automate the execution of MapForce data transformations and other tasks on servers, virtual machines, and workstations, to automate the XBRL data collection and processing. Now the most recent data is available across the entire organization for custom reporting.

You can read the follow-up to the original XBRL case study here.

clip_image001

Read more about how they did it. Read more…

Tags: ,

Versiones en español de XMLSpy, MapForce y StyleVision


Nos complace anunciar que, a partir de la versión 2012 Release 2 SP1, las aplicaciones XMLSpy, MapForce, StyleVision y el conjunto de aplicaciones MissionKit for XML Developers están disponibles en español. También en español está ya el sitio web de Altova.

Estas versiones en español amplían nuestra oferta de versiones traducidas, que ya incluye productos en inglés, alemán, japonés y chino.

Para descargar la versión en español de XMLSpy, MapForce y StyleVision o el conjunto de aplicaciones MissionKit for XML Developers, sólo tiene que seleccionar la opción “Spanish/Español” del menú desplegable de selección de idiomas. Con su actual código clave de licencia puede desbloquear cualquier versión traducida del producto que adquirió. Si lo prefiere, también puede descargar una versión de prueba gratis de 30 días de duración.

clip_image001

Spanish Language Versions of XMLSpy, MapForce, and StyleVision

We’re excited to announce that, starting with Version 2012 Release 2 SP1, Altova XMLSpy, MapForce, StyleVision, and the MissionKit for XML Developers are now available in fully localized Spanish language versions! Much of the Altova Web site is now available in Spanish, as well.

Spanish is latest addition to our growing list of localized options including English, German, Japanese, and Chinese.

To download the Spanish version of XMLSpy, MapForce, StyleVision, or the MissionKit for XML Developers, simply select Spanish from the language selector. Your existing license key code will unlock any language version of your purchased product(s). You can also download a free 30-day trial at any time.

Tags:

Resist Data Integration Redundancy


The Internet makes massive amounts of data available for lots of interesting applications. But whenever you design a unique analysis and presentation of information you don’t privately control, you risk that the owner will offer the same view at some point in the future, instantly making your application redundant.

That’s exactly what happened to the Groupon API data-mining project we originally wrote about in August, 2011. Fortunately, the core of our project is a MapForce graphical data mapping. We can quickly and easily tweak the mapping and repurpose it to present an entirely different data set that provides new value.

HTML output from MapForce and StyleVision

Read more…

Tags: , , ,

XPath Enhances XML Reports


In our previous post on Creating Elegant Reports for GPS XML Data we used an XPath expression to select nodes from an XML document for an elevation line chart. You can also use XPath to compute values. Altova StyleVision includes strong support for XPath, and we can write XPath expressions to add interesting information to our GPS XML reports. For instance, we can process the elevation data stored by the GPS device in meters to plot a chart showing the elevation in feet above sea level instead.

Altova StyleVision line chart

Read more…

Tags: , , ,

Creating Elegant Reports for GPS XML Data


In our earlier post on XML for Global Positioning Systems, we mentioned that adventurers and athletes might want to use XML data from their devices to keep a record of their trips, or even training sessions leading up a marathon or other special event. Several colleagues responded by offering example files!

Looking through all this data, we realized that plotting elevation changes over time would show interesting results for many activities. We used XMLSpy to create this customized line graph directly from the XML data to show elevation vs. time for an afternoon of bicycling through California wine country. We even applied the vineyard photo as a background image right from the XMLSpy chart settings dialog.

Line chart generated by XMLSpy

Whenever you want to elegantly present data from multiple XML data files based on the same XML Schema, Altova StyleVision is the tool that lets you design a richly featured stylesheet for repeatable output in HTML, RTF, PDF, or Microsoft Word formats. Here’s how we did It for our GPS XML data:

Read more…

Tags: , , , ,

Transforming a Plain Layout in StyleVision


Altova Online Training’s newest course, XSLT Technology, leads the student though several transformations of XML data into new formats. Even with all the transformations performed in the class, there are some transformations that did not make it into the final edit.
In Chapter 4 of the XSLT Technology training, I walk students through the process of creating a flyer for the play “Hamlet” using StyleVision. The source content is in XML, and StyleVision autogenerates the XSL stylesheets required to render it into HTML, PDF, and Word, based on our design.

I asked the Altova Art Department to spruce up my design, and, while I did not end up using the nicer design in the training, I wanted to show how to turn my plain flyer into the Art Department’s vision.
clip_image001
Let’s explore the transformation.

Read more…

Tags: ,

FlowForce Server Beta 2 is Now Available


FlowForce Server Beta 2 FlowForce Server, the new server product from Altova that automates execution of MapForce data transformations, has gained hundreds of enthusiastic followers since the release of FlowForce Server Beta 1 last December. Now we are pleased to announce the release of FlowForce Server Beta 2, offered as a free public beta test to all licensed users of Altova MapForce 2012 Enterprise Edition and MapForce 2012 Professional Edition.

The beta test period for FlowForce Server Beta 2 is extended until September 1, 2012.

FlowForce Server Beta 2 adds robust access control to segregate jobs and related data files, so departments can work independently without seeing or overwriting each other’s data. Access control functionality includes defined Users and Roles, Privileges, and Credentials, all managed by FlowForce Server Administrators.

FlowForce Server Beta 2 also includes Web interface enhancements to provide more status information about job schedules, active triggers, and more.

FlowForce Server Beta 2 is available in a 32-bit version and a 64-bit version. The new 64-bit version allows the MapForce Beta 2 module to process very large data files and provides compatibility with 64-bit database drivers.

Click here to read more about FlowForce Server Beta 2 at the Altova Web site, or visit the FlowForce Server Beta 2 download page to get a copy of the installers for yourself!

Tags: , ,

Come Visit Altova this Spring at FOSE!


clip_image002 Stop and smell the cherry blossoms, then come visit Altova while we are in Washington D.C.! We will be at FOSE April 3-5 in booth #719 at the Walter E. Washington Convention Center showcasing the tools in the Altova MissionKit.

If you are attending FOSE, stop by our booth to see the latest features in the Altova MissionKit 2012r2, which include EPUB editing and validation in XMLSpy, sorting of data mapping results in MapForce, generation of code from UML sequence diagrams in UModel, RichEdit functionality in StyleVision and Authentic and much more.

clip_image004 When you stop by our booth, don’t forget to enter our drawing to win a free Altova MapForce Basic Edition license.

Tags: ,

Visit Altova at DevConnections!


clip_image001Join us in Las Vegas as we kick off the 2012 tradeshow season! The Altova team will be in booth # 215 in Las Vegas on March 26-29 at the MGM Grand for the Spring Microsoft DevConnections event.

We would love to see you there, where we will be showcasing the entire suite of tools found in the Altova MissionKit. Altova MissionKit 2012r2 products provide specialized functionality for Microsoft developers and IT pros, including seamless integration with Visual Studio®, C# and C++ code generation, powerful functionality for SQL Server® database management and reporting, integration with SharePoint® Server, functionality for working with Open XML (OOXML), diff/merge for Word documents, and much more.

clip_image003 Feeling lucky? If you are planning on attending the show, don’t forget to stop by our booth and enter our drawing for a chance to win a free Altova MapForce Basic Edition license.

Tags: ,

Building Web Pages – HTML Design with StyleVision


The rapid pace of today’s business environment means that information – along with the format in which it is required – changes often. Although some Web pages contain content that doesn’t often change (e.g., About Us and directions pages), the majority of today’s corporate Websites are continually updated with new data.
For this reason, many organizations choose to store Web content in XML. This allows organizations to develop content in a highly efficient manner because information in the XML file can be used for multiple purposes and in multiple output formats – the XML Schema associated with the XML file describes the content model.
StyleVision is a powerful stylesheet and report designer that can help you leverage XML. StyleVision will allow you to build Web pages with sophisticated formatting in a template-based, drag and drop design window. StyleVision auto-generates XSLT stylesheets so that you can integrate your design into a new or existing site – you can even generate ASPX Web applications right from the File menu.
clip_image001

In this post we’ll design a Web page that will show off some of StyleVision’s HTML formatting capabilities. Although StyleVision’s built in formatting capabilities allow you to create sophisticated designs via simple drag-and-drop, for this example we’ll use CSS3, images, and other standard design elements to create a Web page that doesn’t need to be reformatted when content changes.

Read more…

Tags: , ,

The Global Positioning of XML


XML is everywhere. If you don’t believe it, get a USB cable and connect a GPS device to a computer, then browse for any file with the extension .gpx. We quickly found a 2 MB file named Current.gpx on a Garmin GPS. Opening it in XMLSpy reveals XML data and an XML Schema assignment at the top.

XML Schema assignment in an XML file

Scrolling through the body of the file displays numerous <trkpt> tags that record latitude and longitude along with <ele> and <time> tags reporting the elevation, date and time at that location.

View of an XML file in XMLSpy

Any place you may go in the world, XML is there to tell you where you are.

Read more…

Tags: ,

New! XSLT Technology Training


We’re excited to introduce our new XSLT Technology training as the latest Altova Online Training offering. As with all our trainings, XSLT Technology is released as a free, self-paced course, available online, so students can fit it into their busy schedules.

XSLT transforms XML data into other formats, and this course will transform a beginner XML student into an advanced user. Intermediate and advanced students will gain valuable techniques to add to their XML toolkits.

clip_image002

Read more…

Tags: , , , , , , , , ,

Release 2 of the Altova MissionKit 2012


It’s an exciting week here at Altova! We have several new offerings to tell you about, including Release 2 of the Version 2012 Altova MissionKit and entire product line, a new Altova Online Training course on XSLT Technology, and a new service called Altova Product Consulting. clip_image001 Let’s start with what’s included in the latest product release. Version 2012 Release 2 delivers a host of the new features that have been most frequently requested by our customers, including support for EPUB 2.x, sorting of data mapping results, RichEdit functionality for Authentic eForms, code generation from UML sequence diagrams, and much more.

Version 2012 Release 2 Highlights

EPUB As the demand for e-books and other digital publications has exploded, so have the number of requests we’ve received from developers who need an easy way to create and edit these documents in XMLSpy 2012. In Release 2 we’ve added support for EPUB® (electronic publication), the open standard for e-books from the International Digital Publishing Forum (IDPF). It’s easy to view and work with the various files that make up an EPUB document in XMLSpy’s Archive View, which supports creating, editing, validating, and previewing EPUB digital publications. clip_image003 XMLSpy already provides validation and support for editing XML, XHTML, CSS, and other standards-based files that comprise EPUB documents, making it a natural choice for creating and working with EPUB content. Check out all the new features in XMLSpy. Sorting for Data Mapping Projects Another frequently requested feature is the ability to sort data in mapping projects where the final output format needs to be ordered differently than it was in the input. MapForce 2012r2 includes two powerful new sorting features that you can apply in a variety of ways. The screenshot below shows a new function that performs a multi-level sort on the source data before writing the output file: clip_image004 Another option is available for sorting relational input. MapForce users have long been able to sort database data through ORDER BY statements in SQL queries. However, SQL queries that operate inside the database are not always sufficient for complex data mapping tasks. MapForce 2012r2 adds additional database sorting functionality to the existing MapForce SQL-WHERE component to become SQL-WHERE/ORDER, which is especially useful when database input requires additional processing that cannot be accomplished in a SQL query, or other data or conditions can impact the sorted order of data rows. clip_image005 Read more about new MapForce features. Other important features added in R2 include RichEdit functionality for Authentic eForms created in StyleVision, code generation from UML sequence diagrams in UModel, support for logical files in IBM® iSeries® databases across the product line, and much more.

Upgrade Info

Check out the rest of the features added in Release 2. This new version is free to download and install for customers with Support and Maintenance. If you’re not already an Altova customer, you can download a free, fully functional 30-day trial.

XSLT Technology Training

We are thrilled to announce the latest Altova Online Training course, XSLT Technology. This free, online, self-paced class introduces students to the concepts of XSL, XSLT, XPath, XSL-FO, and more, through guided tutorials and videos. You can practice working with these technologies using the tools of the Altova MissionKit to transform XML to XML, to HTML, and to PDF. clip_image007

Altova Product Consulting

Have you ever wished you could purchase a few hours of a product expert’s time – whether you need to determine if a particular tool could solve your problem, or need help creating a schema or mapping project, or simply require in-depth product support? We’ve heard from a lot of customers with this very requirement – and in response we have created Altova Product Consulting. Altova’s Product Consulting engineers are experts on Altova software and related technologies. Services include determining if an Altova product meet the needs of a particular project, creating a MapForce mapping file (.mfd), designing a StyleVision design (.sps), and so on. Unlike typical complicated consulting engagements, Altova Product Consulting services are sold per hour, the allotment of which is agreed upon in advance, so there are no surprises. Get more information on specific services provided, pricing, and how to get started.

Tags: , , , , , ,

Model Driven Architecture with Altova UModel


For Version 2012, UModel introduces Model Driven Architecture (MDA), with platform-independent models and a Model Transformation feature that transforms all code relevant modeling elements to and from UML, C#, Visual Basic, Java, databases, and XML Schema. Model Transformation A Model Driven Architecture approach to software engineering with platform independent models provides two primary advantages:

  • During the design phase, developers do not need to be concerned with the details and variations between software languages
  • An existing UModel project can be transformed from one source code language to another. For instance, a UML model for a C# application can become a Java or Visual Basic project

Users can even apply model transformation to projects that were reverse engineered from existing source code. For instance, an existing Java application can be reverse-engineered by UModel then transformed to generate Visual Basic classes, and many other possibilities are available. Model Transformation dialog in Altova UModel Platform Independent Models Model Driven Architecture is a set of standards and methods for applying the UML (Unified Modeling Language) administered by the Object Management Group. In Model Driven Architecture, the UML model of a software project is a platform independent model (PIM) that can be fully described without concern for the details of any specific programming language. This development strategy allows software architects and other developers to focus exclusively on logic required by the subject domain, rather than characteristics of any programming language. Data Type Mapping During model transformation, UModel maps data types from the source to the target to accommodate differences between languages. The Type Mapping dialog lets you review or even edit type mapping pairs. Type mapping for UML model transformation UModel also automatically adds the target language profile to the transformed project. UML Class Diagrams As part of the model transformation, UModel creates new UML classes and class diagrams for the target, reflecting classes and class diagrams in the original project. The screen shot below shows the Hierarchy of Account diagram for Visual Basic after model transformation from Java. The new Account class in the new folder named VB Target in the model tree contains Visual Basic syntax for all properties and operations. For instance, the new balance property is defined as the Visual Basic Single data type, whereas in Java the data type was float. After transformation, the original Account class for Java is preserved in the model in its original location in the model tree. The original UML design for Java will now generate code in multiple source code languages – Java and Visual Basic. UModel class diagram and Model Tree Persistent Transformation Parameters The transformation paradigm extends to updating existing transformations and merging the updates into the specified target models. Transformation parameters are stored in a Model Transformation Profile in the model. The Transformation Profile can be set to run transformations automatically before forward engineering code generation, and/or after reverse engineering, to update elements for one target language based on changes to model elements for another. UModel transformation parameters stored with model These Transformation Profile settings can also be changed at any time. UModel Transformation Profile settings This functionality lets UModel automate much of the maintenance of multiple source code languages as your cross-platform model evolves. If you’d like to try out Model Driven Architecture and model transformation with UModel 2012, you can download a free 30-day trial.

Tags: , , ,

Data Exchange for the Mobile Workforce


Data Exchange for the Mobile WorkforceOrganizations have been forced to adapt many of their internal business processes to accommodate an increasingly mobile workforce. Although there are technological solutions that address many of today’s communication needs, the plethora of different document formats in use – even within the same organization – means that some tasks remain vexing. For example, how does an organization remain flexible enough to facilitate the exchange of data among mobile workers yet retain the ability to bring that data into internal IT systems? Altova offers an inexpensive solution with StyleVision®, a graphical stylesheet and report designer with electronic forms capability, and Authentic®, a WYSIWYG XML and database editor in which end users can view and edit electronic forms created in StyleVision. The Authentic Community Edition is available from the Altova Web site as a free download so that anyone can be brought into your workflow, whether they are internal or external to your organization.clip_image002In this post we’ll present a sample case showing you how to create an electronic form that mirrors an existing paper form (in our example it is a reimbursement form) and then we’ll follow it as it makes its way around a fictional organization. Please note that the example we use here is simplistic and was designed only to illustrate the process of developing and deploying an electronic form. Although you can add additional data sources and perform validation and other complex functions in StyleVision, we have not illustrated these here.It is extremely easy to design electronic forms in StyleVision. To start, we simply select New – New from XML Schema/DTD/XML … from the File menu, browse to an XML Schema file, and select the type of design we’d like to create. For this example we created an XML Schema and instance file in XMLSpy, Altova’s XML editor and development environment, based on the fields on the paper reimbursement form. You can also base a StyleVision design on a database or XBRL taxonomy.Below is a copy of the reimbursement form we will be using along with the XML Schema we created.clip_image002clip_image003Once we select the XML file in StyleVision, we are prompted to select either a free-flow or form-based document. In a form-based document all design elements (e.g., text boxes for user input, images, buttons) are fixed in position – ideal for data entry forms.When we create a form-based document, we can upload a “blueprint image” so that we can recreate a paper-based form exactly as it was originally designed. This is the option we’ve selected below. The image will appear in the background of the design window and we will simply place design elements on top of corresponding elements on the form. Of course, the blueprint image overlay does not appear in the final output.clip_image004The screenshot below shows the blueprint image as it appears in the design window in StyleVision – how cool is that? clip_image005Now, using the Insert menu at the top, we can simply insert design elements onto the blueprint image in the design window.Available design elements include form controls (e.g., input boxes, combo boxes, radio buttons), images, tables, charts, and “layout containers” for exact positioning.We’ll start by adding input fields to capture employee information (i.e., First [Name], Last [Name], Title, etc. from the top part of the form). Once we click Insert – Insert Form Controls and select Input Field, the Insert Design Element dialog box appears.We have highlighted the First [Name] element in the dialog box below – the input field will now be associated with the First element. This way, when the end user types data into the input field and saves the form, this information will populate the First [Name] element in the XML file.clip_image006We now add design elements throughout the rest of the form, associating input fields with their respective elements from the XML file.When we are finished adding input fields, a logo, lines, a table for the expense items, and labels, the design looks like this – we’ve set the opacity attribute for the blueprint image to 0 to make it easier to view the design elements. Please note also that we’ve done some additional design work such as adding calendars in date fields, drop down boxes, and a currency sign that changes according to user input. For more information about fine tuning your form please see the StyleVision User and Reference Manual in the StyleVision application.clip_image007We can preview how the end user will see the form we designed in StyleVision by clicking on the Authentic eForm tab at the bottom of the design window (below). Note that the end user is prompted to enter data directly into each data input field. We accomplished this by placing the prompts (e.g., Insert First Name) in between the relevant tags in the XML file associated with our design. The end user simply highlights the prompt and replaces it with text.clip_image008Once we are finished designing the reimbursement form we can save the entire design – including the XML Schema and instance files, images, and any other associated files – in a single PXF® (Portable XML Form®). Saving the design as a PXF will enable us to email the form along with data updated in the underlying XML form among people both inside and outside the company’s LAN.clip_image009Once we hit OK we are prompted to select the files to include in the PXF. Notice that we’ve selected HTML, RTF, PDF, and Word 2007+ under the Generate and store XSLT files … heading. This will allow an end user to generate the form – with data – in these formats directly from Authentic.clip_image010Now that we’ve saved it in a PXF, the electronic form we designed in StyleVision is ready to be deployed in a business environment.In our example, we have a team of salespeople working across the globe who need to request reimbursement for business and travel expenses incurred. The salespeople complete expense reports, forward them to their managers for approval, and then send approved reports to the corporate office so that the information can be added into the accounting system.The PXF makes this easy.Once a salesperson is ready to complete a reimbursement request, she simply opens the PXF in Authentic and can immediately begin entering information onto the form. Below is a screenshot of a reimbursement form that has been completed in Authentic – notice that the form still needs a manager’s signature.clip_image011Now the salesperson must send it to her manager for approval. It’s easy to initiate an email with the form attached directly from Authentic. clip_image013Once the manager receives the email, she can simply double click the attachment and it will open in Authentic. Here the manager has clicked the Approved check box and added her name and the date.clip_image014The manager can then email the updated PXF back to the salesperson, who in turn emails it to the corporate office so it can be imported into the accounting system for processing. Our fictitious corporate office of course receives hundreds of reimbursement requests each day and has established a process for importing them into the relevant Oracle databases in the accounting system.We’ll use Altova MapForce, a graphical any-to-any data mapping, conversion, and transformation tool, to populate the corporate database with the data from the quotations. After setting up the mapping, we’ll automatically generate code from MapForce so that we can automate the transformation either through batch processing or a real-time conversion.First we’ll set up the mapping.We’ve inserted the XML file ExpRpt which we’ve extracted from the PXF into the left side of the MapForce design window and then inserted the Oracle database on the right side of the design window.Now we can drag and drop fields from the XML file with the reimbursement data into the Oracle database. We can also transform data, as we’ve done with the Approved element. Here we’ve used the built-in boolean function to convert the string value stored in the XML file (“true” or “false”) into the numeric equivalents (1 or 0). We can also create our own functions.The mapping we’ve created appears below.clip_image015Please note that this post offers a very broad overview of how to use both StyleVision and MapForce. Please visit the online training section of the Altova website for more in-depth instructions on how to use these and other Altova products.And there you have it. With the PXF, the fictitious Nanonull Corporation allows a group of far flung sales reps and their managers to easily exchange and edit information via electronic form. The PXF also provides a way for Nanonull to populate the accounting database without offering these employees direct access to company IT systems. All without busting the IT budget.

clip_image016

What could your organization do with a flexible, portable interactive document? Please share your ideas with other users by commenting on this blog post. Have you used StyleVision or other Altova products in an interesting project and think it would make a great case study? Email us at marketing@altova.com. We’d love to hear from you!

Tags: , , ,

Intuitive User Interface Features


Here at Altova we like to share user interface features across products to give the MissionKit a consistent look and feel, and to help users leverage experience gained using one tool to get up to speed more quickly with the others. In version 2012 we introduced new visual alignment guides in both UModel and MapForce to make it easy to work very rapidly, yet still produce a neat, organized result that communicates effectively with other team members. As users drag elements in the diagram window, alignment snap lines appear automatically to allow any component to align with any other component. UML class diagram in Altova UModel In the UModel class diagram shown above, the developer has collapsed the properties and operations compartments to concentrate on class associations. As the CheckingAccount subclass is dragged upwards on the screen, a visual alignment guide appears and offers instant snap-to alignment with the other subclasses of the Account class. Complex data mappings in Altova MapForce might include dozens of components, functions and constants, and visual alignment guides can help organize the mapping view. In the screenshot below, the developer used alignment guides to collect split-name functions together, while concat functions are aligned in a separate group. Altova MapForce data mapping Constructing the mapping diagram this way can greatly clarify the developer’s intent when multiple complex string manipulations are needed! Of course, aligned components also improve the value of printed diagrams that become part of the permanent documentation for UModel and MapForce projects. We also understand that not everyone likes to work the same way. If you prefer not to see guide lines and to turn off automatic alignment, this feature can be deselected in the Tools / Options menu selection in both UModel and MapForce. If you’d like to see for yourself how intuitively tools in the Altova MissionKit 2012 work together, click here to download a free trial!

Tags: , , , , ,

Get More Mileage from Your StyleVision Designs with Dynamic Selection of CSS Files


One of the great things about StyleVision is the flexibility that it affords. With StyleVision you can design visually sophisticated stylesheets and reports for output to multiple formats including HTML, PDF, and Authentic electronic forms. Now StyleVision makes it even easier to accommodate different formatting needs in a single design with dynamic selection of CSS files – a new feature introduced in StyleVision 2012.

Although you can still manage virtually all formatting tasks from the StyleVision design itself, dynamic selection of CSS files means that you can create a single template for use in multiple situations. To demonstrate, we’ve created an invoice for the fictional Nanonull Corporation for which the design changes based on the number of days that have elapsed between the invoice issue date and the due date. Although we could have used XPath to change the formatting of each element based on the number of days that have elapsed, in this case it is more efficient to create individual CSS files that apply to different scenarios.


Below is the invoice that we designed in StyleVision. Notice that beyond the bold headings it is almost devoid of formatting – we’ll be applying formatting based on the number of days that have elapsed between the invoice issue date and the due date via CSS files.

Now we can create our CSS files in XMLSpy, Altova’s XML editor with integrated support for editing CSS2 and CSS3.

First we’ll create a CSS file for invoices with due dates after the invoice issue date (i.e., invoices that are not overdue). Here we’ve simply created the body selector with a yellowish background and the Nanonull logo and a header class.

Now we’ll create a more elaborate CSS for overdue invoices. The body selector will include a past due image, the paragraph selector will have 20 point bold red font, and the header and small header classes will have red bold font in different sizes.
Now we can assign the CSS files to the StyleVision design and subsequently assign classes to different design elements.
In StyleVision, we simply click add new CSS file in the Design Overview and navigate to the CSS files we created in XMLSpy – StyleVision supports multiple CSS files in the same design.
Our design now looks like this – notice that formatting from both CSSRegular and CSSOverdue have been applied in StyleVision design view.
However we would like to assign the CSS file based on the distance between the invoice issue date and the due date. Therefore, we’ll right click on the arrow next to the CSS file in the Design Overview and choose Select One CSS File with XPath (below).
This will invoke the Select CSS File Using XPath dialog box (below). We can use XPath in this dialog box to select the appropriate CSS file based on dynamic input – here we’ve instructed StyleVision to use the CSSOverdue.css file if the invoice due date precedes the invoice issue date and the CSSRegular.css file in all other cases.
Here is the complete XPath we entered into the Select CSS File Using XPath dialog box above.
Now we can assign the classes defined in the CSS files to individual design elements in the StyleVision design.
Here we’ve assigned the paragraph containing the text “Invoice” to the class “header “ in the Styles pane – based on the contents of our CSS files this text will appear in red 20 point bold font for overdue invoices and black 20 point bold for invoices whose due date is in the future.
Likewise, we’ll assign the invoice due date to the class “small header.”
Now we’re ready to preview the invoice.
Below is the HTML preview for an overdue invoice – note that the text “Invoice” and the due date are in red bold font and that Past Due image is stamped in the middle.

Now let’s look at the same invoice with a due date in the future – note the yellow background, Nanonull logo, and that “Invoice” and the due date appear in black.

The ability to select a CSS file based on dynamic input opens up a wealth of possibilities and makes your StyleVision designs even more adaptable. With this feature, you can make formatting changes on the fly via the CSS files associated with a design – especially handy for designs with many elements.
Created a great project using the StyleVision stylesheet and report designer or any of Altova’s other tools and want to show it off? Please share your story with other Altova users by commenting on this blog post.
If you think it would make a great case study please visit the Altova case studies page – if we use your story you’ll receive a $200 Amazon gift card. We’d love to hear from you!

XML Editor Supports HTML5 and CSS3


The Altova MissionKit Web tools received an important update with our recent Version 2012 release: support for HTML5 and CSS3. You’ll find support updated to include the most recent versions of these Web standards in both XMLSpy 2012 (and higher) for code editing and StyleVision 2012 for graphical stylesheet and report design.
clip_image001
Let’s focus on the HTML5 and CSS3 editors in XMLSpy here – StyleVision functionality will be covered in a future article.

HTML5 Editor

XMLSpy has long supported editing of HTML4 and, of course, XHTML – and now those intelligent coding features extend to HTML5.
If you’re already an XMLSpy user, you’ll immediately recognize the helpful syntax coloring, source folding, and line numbering in advanced Text View when you open your HTML5 file for editing. As you type, you’re presented with valid HTML5 element and attribute choices in a drop down menu as well as in static entry helper windows, so you can complete code in your preferred working style. At the same time, code completion helps you work faster while ensuring elements are closed properly.
clip_image002
These intelligent editing features are applicable to the version of HTML you’re working with – when you open an existing HTML5 document or start a new file from scratch, all the HTML5 elements and attributes are immediately available in the entry helper windows and drop down menu. Some of the most notable new elements in HTML5 include:

  • <video> and <audio> for media playback
  • <canvas> for dynamic rendering of 2D images
  • <article>, <section>, <header>, <footer>, and <nav> for including richer semantic information to describe content
  • <calendar>, <date>, <time>, and other form control elements
  • And so on

XMLSpy includes an integrated Browser View that lets you see the results of your HTML5 coding immediately. The Browser View renders your page using your installation of Internet Explorer directly in XMLSpy…
clip_image004
…but also allows you to immediately view your HTML5 Web page in an instance of any browser you have installed on your machine.
clip_image005
This allows for quick testing and debugging to ensure cross-browser compatibility.
To get an idea of how some of the new HTML5 features look in action, check out the simple HTML5 example document that is supplied in the XMLSpy 2012 Examples folder and linked from the Example project. To learn more about HTML, there are many excellent resources on the Web, including  http://www.w3.org/TR/html5/.

CSS3 Editor

Working hand-in-hand with the HTML5 editor is the XMLSpy CSS3 editor, which also provides syntax coloring and entry helpers, as well as bookmarking and source folding in Text View.
clip_image006
A CSS Outline window displays an outline of the document organized by its selectors listed in groups. Each group can be collapsed and expanded, and clicking a selector in the CSS Outline highlights it in the document. CSS Properties and HTML elements entry helpers are shown in windows as well as context sensitive drop-down menus, and XMLSpy even includes screen tips that provide a definition of each property and its possible values as you type.
clip_image007

You can take advantage of XMLSpy’s integrated Browser View here, too, to see the results of your CSS edits immediately in a linked HTML page.

clip_image008

Why XMLSpy for HTML?

Sure, XMLSpy is known for all things XML, but it’s also a clear choice for HTML development. HTML5 and CSS3 editing in XMLSpy is facilitated by support for related technologies including XML, XSLT, WSDL, and others, as well as integrated project management functionality, support for popular version/source control systems, and Microsoft® SharePoint® Server integration – giving you all the tools you need to develop the next generation of Web sites and apps.

We’d love to hear how you’re using HTML5 now – please drop us a line by leaving a comment below.

Tags: , , , ,

New in StyleVision 2012 – Composite Styles for Enhanced Formatting


StyleVision is an intuitive stylesheet and report designer that transforms XML, XBRL, and database content into HTML, RTF, PDF, Word 2007+, and Authentic electronic forms – all from a single design. With the 2012 release (available for download here), StyleVision now offers even more formatting control so that you can design even more visually sophisticated output.

One of the more versatile features introduced in StyleVision 2012 is support for composite styles for output to RTF, Word, and PDF as well as HTML and Authentic electronic forms. This feature allows you to combine styles defined in an XML instance document with those you set in the report itself. You can also build a composite with XPath to control multiple style features of the same design element (e.g., font, background color, alignment).

Support for composite styles means that you can quickly and easily change the look of a design component by selecting an attribute from the XML instance file or by changing the XPath expression. This feature is used most often in cases where the XML document includes HTML fragments that contain style information – now you can use an XPath selector to extract style information from the HTML fragment and apply it directly to an output document.

Here for example we’ve used XMLSpy, Altova’s XML editor and development environment, to create an XML instance file containing the contents of a music library. (We also created the Schema in XMLSpy.)

The StyleIt attribute of the Recording element pictured below includes formatting instructions for this individual recording – according to our file the Fun Factory recording should be red and bolded.

Now we can move to StyleVision and start creating a design based on our XML instance document.

All we need to do to apply the formatting instructions in the XML instance document is to call the StyleIt attribute from the composite style drop down in the Properties pane in our StyleVision design.

The Fun Factory recording appears in red bold in the resulting output.

We can also choose to ignore the formatting instructions from the instance file and provide our own.

Here we’ve replaced the StyleIt attribute in the drop down list with an XPath if-then-else statement that assigns formatting based on an attribute of the Recording element called PlaylistCode – party tunes are in blue Papyrus 12 point font, easy listening favorites are in maroon Pristina 14 point font, and background music by default is in green Onyx 14 point font.

Here’s what the output file looks like now. 

Support for composite styles is only one of the new styling features in the StyleVision 2012 stylesheet and report designer. Click here to read more about the new features across the entire Altova product line.

Have you used created a great project using the StyleVision stylesheet and report designer or any of Altova’s other tools? Want to show it off? Please share your story with other Altova users by commenting on this blog post. If you think it would make a great case study please visit the Altova case studies page – if we use your story you’ll receive a $200 Amazon gift card. We’d love to hear from you!

Announcing Altova FlowForce® Server Beta 1


Altova FlowForce® Server Beta 1 is an exciting new tool for execution of automated data mappings designed to provide comprehensive management and control over data transformations performed by dedicated high-speed servers, virtual machines, or even regular workstations, depending on the size of the task. To gather user input and accelerate product development, Altova is offering Altova FlowForce Server Beta 1 as a free public beta test to all licensed users of Altova MapForce® 2012 Enterprise Edition and MapForce® 2012 Professional Edition. Users employ Altova MapForce data mappings for two different types of data transformations:

  • One-time data conversions
  • Data transformations that are repeated by date or time, when new source data is available, or based on some other external event

While command-line execution, royalty-free code generation, and the MapForce API can assist with automation of repeated transformations, FlowForce Server Beta 1 provides much greater power and flexibility. FlowForce Server Beta 1 is a server-based tool with a Web interface that makes it much easier to implement, manage, or modify data transformation jobs in a busy data processing environment. FlowForce Server Beta 1 can administer multiple transformation jobs simultaneously, lets users define and adjust a variety of job triggers and actions on the fly, can perform housekeeping tasks like moving output files or cleaning up intermediate work, records detailed logs of all activity, and much more. FlowForce Server Beta 1 consists of four components that work together as illustrated in the diagram below. Altova FlowForce Server Beta 1 block diagram The FlowForce Server continuously checks for trigger conditions, starts and monitors job execution, and writes detailed logs. MapForce Server is an implementation of the MapForce Built-in execution engine that executes mapping packages previously deployed via the MapForce graphical environment. The FlowForce Web Administration Interface is a standalone web application that runs in an internet browser and provides the front-end of FlowForce Server. MapForce Beta is an enhanced version of the Altova MapForce application with an integrated deployment feature to deploy MapForce data mappings to a FlowForce server package. Altova FlowForce Server Beta 1 is available immediately as a free public beta test to all licensed users of Altova MapForce® 2012 Enterprise Edition and MapForce® 2012 Professional Edition. Visit the FlowForce Server Beta 1 page at the Altova Web site for more information!

Tags: , , ,

Processing the Groupon API – Epilogue


Rare edge cases can derail loosely coupled data mapping applications. This is especially true when you are consuming large datasets available over the Internet and have little or no influence over the source data. In this article we describe a debugging technique that lets developers working on data mapping and transformation projects quickly identify and accommodate unexpected data in a stream from a remote source. The Problem Last summer we wrote a series of blog posts describing how to work with the Groupon API to retrieve a subset of offers in all Groupon cities and format the list for a web browser or mobile device. MapForce output from the Groupn API, displayed on a mobile device We concluded with a command line to run a MapForce data mapping that calls the Groupon API over 150 times — once for each Groupon city, then filters the data to extract deals sold on the Internet instead of a physical location, and formats the results in HTML using StyleVision. Every morning we run the command line in a batch file that saves the HTML output on a local server so our colleagues can check it out with any Web browser to find interesting offers from all over the country. The mapping ran fine for more than two months until one day it failed with this error message: “Source-value “” of type dateTime could not be converted into target-type dateTime.” The specific explanation is that somewhere in the mapping where we expected a dateTime, we received an empty value. On a more abstract level, the error suggests a potential defect in the logic of our mapping strategy. Every time we call the Groupon API we receive a well-formed XML data stream enclosed in a <response> element, but the API specs do not include an XML Schema defining the data that may be returned. When we developed our mapping we needed to analyze the raw data and select the output we wanted, so our first step was to call the API to capture all the Groupon deals for one large metro area. We assumed we would get a large enough data sample to include every possible option in the API response. After our mapping ran successfully for two months, the API finally delivered a rare edge case that did not fit the pattern we expected. Debugging Tools MapForce provides debugging help. We can run our data mapping using the MapForce built in execution engine to see more details in the Messages window. MapForce Messages window siplays data mapping error The lines labeled Related location are hyperlinked back to components in the mapping where the error occurred. Clicking on the result error takes us to a format-dateTime function. format-dateTime function in MapForce We can either click the “” error or trace the value connector to identify the input element to the format-dateTime function. Either way, we locate the element that triggered the error. clip_image004 The suspect element resides in the input component that captures all the data returned by our calls to the Groupon API before any filtering or conversion takes place. When we designed the mapping, the endAt element in our sample data always reported the ending date and time for each Groupon offer, but for some reason we must have received an empty value in this field. If the error had occurred by running a local input file we could simply examine the file contents, but in this case the data came from multiple URLs, and is only held temporarily until it is mapped to the output component. Fortunately, we can apply a trick to easily modify the mapping and preserve all data received from the Groupon API. We simply copy the input component and paste a duplicate into the mapping. We can connect the response element from the original to the duplicate, which simultaneously maps all the child elements between the components. clip_image005 Our original input component is now connected to two output components. We can select which output component will be generated by the MapForce built-in execution engine by clicking the eye icon at the top right corner of any output component. The new output component simply saves a copy of everything in the input component. When we examine the raw data using XMLSpy, sure enough we find an empty element where we expected a date and time: clip_image006 The Solution Now that we know an offer might have no specific end time, we can plan for that possibility in the mapping. In the revised treatment of the endAt element, we do an if-test before the original format-dateTime function and provide an alternate output when the endAt element is empty. clip_image007 We had to work fast because all Groupon data is time sensitive. The edge case would eventually expire and disappear from the data stream. This experience showed us how important it is to have powerful debugging tools and to use them creatively, even after you think a data mapping project is running successfully! Altova MapForce is available in a free trial – the next edge case you solve could be your own. Editor’s Note: Our original series on mapping data from the Groupon API ran in three parts you can see by clicking the links here: Part 1 of Processing the Groupon API with Altova MapForce describes how to create dynamic input by collecting data from multiple URLs. Processing the Groupon API with MapForce – Part 2 describes how we filtered data from the API and defined the output to extract only the most interesting details. Processing the Groupon API – Part 3 describes formatting the output as a single HTML document optimized for desktop and mobile devices, and reviews ways to automate repeat execution.

Tags: , , , ,

Analyze Football Statistics using the Altova MissionKit


In this article we use stats from NFL.com and ESPN.com to show how easy it can be to process and analyze online data in new ways – even when it uses different metrics and is only available in textual format. We have seen in previous blog posts how easy it is to gather data from the Internet that is widely available in XML formats. But what about interesting data that is available online but not in an XML format, or data that is buried in legacy data processing systems and only available in textual report format? One such example involves quarterback ratings. The NFL has used a Passer Rating that rates quarterbacks solely based on a passer’s completions, attempts, touchdowns, and interceptions. ESPN introduced a new rating system this year called the Total QBR (Quarterback Rating). The Total QBR incorporates more data, including an expected points average and a clutch play index, that ESPN claims gives a more accurate measure of a quarterback’s performance. Let’s compare the rankings that these system produce to see if we can garner some useful information. For this example we’ll be using the data importing and analysis tools of the Altova MissionKit to compare the ratings. If you want to try this out yourself, the MissionKit is available to download for a 30 day free trial from the Altova web site. You can access the files used in this example here. The first thing we need is the raw data to analyze. Let’s use the entire 2010 season as a data source. We can get the table with Passer Ratings from NFL.com and then copy and paste it as a new text file. NFL.com_top5_passers_2010 We can access a similar table of Total Quarterback Ratings from the ESPN web site and create a second text file. ESPN_Total_QBR_Top5_2010 We now have two text files with tables of data in different orders. The next step is to combine the tables into one file and generate charts. First, we need a schema file for the destination of the data. In XMLSpy, we can create an XSD file quickly, and graphically, to contain a series of QB nodes with child nodes of first and last name, team, passer rating and rank, and total QBR and rank. QB_Schema.xsd Now, in MapForce, we open the text documents and use FlexText to parse the text and change it into a list of categories. NFL_QB_Data_FlexText Total_QBR We then build a mapping file in MapForce to map the data from the text files to the destination XML file. Built-in functions make it easy to extract the first and last names from the Player string, and a value-map will change the team abbreviation to a string (ARI is changed to Arizona Cardinals, ATL to Atlanta Falcons, etc.). We set the Priority Context in the test of our filters to make sure we get the correct set of data for each unique quarterback. QB_Schema Once we execute the mapping, we can save the resulting XML data file and use it as the source file in StyleVision to design a stylesheet. In this stylesheet, we create a table of the top ten ranked passers and charts showing the Passer Rating and the Total QBR graphically. QB_Charts1 QB_Charts2 Now that we have a visual representation of the rankings of the two rating systems, we can examine their differences and try to see which works better. For example, Peyton Manning was tenth in passer rating, but was second in Total QBR. This can be explained by the Total QBR taking clutch points into account and knowing that Peyton Manning had a few late game comebacks in the 2010 season. Since we now have a collection of files (the XSD file built in XMLSpy, the FlexText and mapping files from MapForce, and the stylesheet design created in StyleVision), we can update the text data files easily to analyze new sets of quarterback data. Later in the season, we can update the text tables with 2011 data, and allow the data to flow through the mappings and into the stylesheet to update the charts and see the rankings for the current season. This example focuses on numbers from the NFL, but this method can easily be adapted to other data sets and data sources that are accessed as text files as well as in other formats. You can learn more about how to use the products in the Altova MissionKit by taking our free online training courses.

Tags: , , , ,

Mastering Paid Keywords


Anyone who manages paid keyword search knows it is hard work! You can look at vast reports of raw statistics and quickly get lost in trivia. At Altova we designed a better way to analyze and manage the performance data for our Google Adwords campaigns. We can creatively query the numbers to: · Quickly aggregate results for subcategories of campaigns, for instance by product, geographical region, or any other grouping · Easily identify trends over time The chart below illustrates these advantages by collecting data for a single Altova product – SemanticWorks – from multiple campaigns over six individual months. Keyword performance chart created with DatabaseSpy Starting Out Like many keyword advertisers, we were viewing statistics in Adwords, downloading CSV files, then spending hours massaging and manipulating the data in spreadsheets to identify and format the information we required. We wanted more immediate and in-depth reporting of keyword performance while retaining full control of the process and managing everything internally. SQL queries of a database of keyword statistics offer a powerful and flexible alternative. In the remainder of this post we explain how the database design, data mapping, and reporting features of the Altova MissionKit can be applied to create an architecture to efficiently track paid keyword performance. Database Design Our choices were to implement a keywords database on an existing database platform already running in the company, an express edition of a commercial database, or an open-source database, since the Altova MissionKit works with SQL Server®, MySQL®, Oracle®, IBM DB2®, PostgreSQL®, Sybase®, and Microsoft® Access®. We chose SQL Server for our database platform. We connected with DatabaseSpy and used the graphical database Design Editor to create the table shown below. DatabaseSpy graphical table design Most columns correspond to fields in a keywords report. In order to store multiple rows for each individual keyword – one row for every month of statistics – the table also includes columns for the month and year. Populating the Table The Google Adwords online interface lets users create reports of keyword statistics of specific date ranges and download them as CSV files. We downloaded individual CSV files containing our performance data for each unique month. We used MapForce to map values from the CSV files to columns in the database table and insert the month and year data for each row. Keyword report mapping in MapForce The string functions at the bottom center of the mapping diagram remove percent signs and commas from fields we want to treat as numerical data. By doing this in the mapping, we don’t have to massage the columns of data in the CSV files before importing them. Since the CSV files for each month all have the same structure, the mapping needs only minor revisions to import each new month’s data: update the constants at the top that define the starting row id, month, and year. MapForce processes the mapping with its built-in execution engine, reading the CSV input and generating SQL INSERT statements for each row of data. MapForce then allows users to execute the entire generated SQL script by clicking a toolbar icon or from a selection in the Output menu: MapForce database insert script Querying the Database Back in DatabaseSpy, we can query the database from the SQL Editor window. This query reports the top ten performing keywords for SemanticWorks in October 2011. For data privacy, some fields in the Results chart are hidden. Results with table To get additional interesting results, the SQL statement can be easily modified. For instance, the ORDER BY line can sort for highest cost, most clicks, or any other characteristic. The WHERE statement combines data from multiple campaigns. The LIKE keyword treats the percent signs around SemanticWorks as wildcard characters to match any campaign with SemanticWorks anywhere in its name. Other queries could add a geographic identifier such as US or EU, or match on an entirely different column such as adgroup. Of course, all these options depend on a consistent and predictable campaign and adgroup naming system. We created a DatabaseSpy Project to collect all our favorite SQL queries for sharing and convenient reuse. Here is the query we used to generate the chart right in DatabaseSpy that appears at the top of this post: ChartQueryCapture This query goes beyond simple SQL reporting to perform calculations on a subset of the data and format the results. Database Reports We designed reports for the executive team using Altova StyleVision, based on the queries and charts we had already designed in DatabaseSpy. We simply copied our queries from the DatabaseSpy SQL Editor window and added them as sources in the StyleVision Design Overview window. Saving our report design in a StyleVision SPS stylesheet makes it is easy to regenerate an updated version every month. Here is the HTML output for a SemanticWorks Keyword Trends report based on the query above, displayed in the StyleVision Preview window: clip_image009 If you follow the conventional wisdom for building your own paid keyword campaigns, you will develop segmented campaigns with many small, highly specialized ad groups, and you may also find yourself overwhelmed by the data in Adwords reports. If you’d like to try managing your own keywords the way we describe here, a fully functional trial of the Altova MissionKit is available.

Tags: , , , , ,

DiffDog Takes to the Cloud


Techy folks generally have a good diff tool they rely on to compare and sync files and directories. But what happens when, as more and more info is bound for the cloud, your data lives on servers accessed via URL? DiffDog diff/merge tool There are myriad applications today that live on servers accessed via HTPP – but let’s take a look at a common example: SVN. Subversion (SVN) repositories include WebDAV as a commonly used server option. WebDAV is a natural protocol for SVN because its concern is hierarchy, structured metadata, and versions. Since WebDAV is an extension of HTTP it gives easy access to basic information about files and folders to any HTTP-aware client, including DiffDog – Altova’s diff/merge tool for files, directories, and databases. However, DiffDog knows a few tricks that set it apart from the other breeds.

Diff/Merge via WebDAV

SVN clients typically support command line differencing; however, a text-only representation of the changes in even one file can be hard to read and use. When you want to compare the trunk against a tagged version, the problem is magnified.  There are several visual differencing tools available that can help with analyzing version changes in SVN. They have varying degrees of compatibility with how SVN works. Some tools are well integrated with the SVN command line. DiffDog includes all the common comparison options for a tool that is tightly integrated with SVN clients.  Where it excels is its ability to talk to SVN servers.  Accessing an SVN repository with DiffDog using WebDAV is simple. The easiest starting point is to open Directory Comparison View and paste in the URLs of the folders you want to compare. In this case we’re comparing SVN branches on Projectlocker.com. The two sets of files open, and DiffDog provides a color-coded, browsable view of the differences between the two directories. Directory Comparison in DiffDog   Clicking on either one of a pair of files opens a detailed file comparison.   File comparison in DiffDog DiffDog’s ability to distinguish between changes to XML and meaningful changes is key in this situation – most development trees have some amount of XML in them.  DiffDog also supports comparing Word docs and databases – so all bases are covered. XML-aware diff options Of course, folders you compare do not have to both be WebDAV SVN folders.  It is equally straightforward to compare the SVN server with a local directory. DiffDog’s ability to access servers via HTTP (or FTP) opens a world of possibilities: comparing a local directory with a Google Docs directory, or diffing a local Web server against files hosted on the Amazon CloudFront , or even just synching photos between your local drive and your chosen back- up service.   If you’d like to try DiffDog, it’s available for a 30-day trial over on the Altova Web site.

Tags: , , ,

Digging deeper with the Twitter API: iPhone 4S vs. Galaxy Nexus


We found some interesting data when we dug below the surface of the iPhone 4S vs. Galaxy Nexus debate using the Twitter Search API.In today’s world there is a vast quantity of data available online that can be used for research, market analysis, and competitive intelligence. While “Big Data” can be a problem for those who produce it, store it, and compile it, it is highly beneficial for those of us who are looking for answers.Some of that data is fortunately available to be queried online, and, in particular, there is a vast quantity of data on social media interactions out there.TweetsQueryingSearchAPIIn this article we will explore how to use the Twitter Search API from MapForce, Altova’s data mapping/conversion/integration tool, to aggregate data on recent user submissions (“tweets”) on two highly popular topics – the Apple “iPhone 4S” vs. the “Galaxy Nexus” as the latest hot Android phone – and extract some statistical data about the users engaged in those discussions. One of the benefits of this abundance of data available to us today is that we can query it in interesting ways and extract new meaning from it. While there are undoubtedly many existing services that already provide trends over Twitter topics (e.g., Trendistic), those services only offer very simple trends and do not allow us to query any deeper.But all of the underlying data is available for grabs if you are just willing to learn a tiny bit about web service APIs and how to use them to extract XML data for further processing. As a starting point, let’s use the Twitter Search API to query the stream of recent tweets for the last 100 postings that are about the “Galaxy Nexus”. The Usage Guidelines for Twitter Search tell us that using both words in a query will result in the use of the default operator, which is AND, so we are going to search for posts that contain “Galaxy AND Nexus”. So let’s try that and request the most recent 100 items:

http://search.twitter.com/search.atom?q=galaxy+nexus&rpp=100

If you follow this link, you will get a second window with a lot of raw XML data that is formatted according to the Atom Syndication Format specifications. Alternatively, you could request the data in JSON format, if you wanted to directly process it via JavaScript code by hand, but we will use the XML-based Atom format so that we can easily analyze the data and extract the information we want.Viewing the above search result in a browser is not very user-friendly, so we can take a quick peek at the XML data in our favorite XML Editor using the Open from URL function:TweetsAtomGridAs you can see, the data for each entry includes a language code, so for this example we will extract data from this Twitter feed as well as from a second search result on the “iPhone 4S” and combine them into one intermediate XML file for further analysis.Extracting XML data is really easy in MapForce: using the “Insert XML File” option to drop in an XML source, we can again specify the same URL as before. If needed, MapForce will automatically create an XML Schema for the supplied data so we can visualize it and extract information from it:TweetAtomMappingIn our mapping we have dropped in two sources on the left side – one using a query string to search for “Galaxy Nexus” and the other to search for “iPhone 4S” – and on the right side we have dropped in a simple XML Schema that will allow us to aggregate our data and analyze it more conveniently going forward. In this case the mapping between the two sides is straight-forward as we are only extracting basic information about the user, the date, and the language of the tweet, but in other applications the mapping could be more complicated and include functions as well as queries to other data sources, databases, or web services…Previewing the resulting XML data can be done directly inside MapForce using the output tab, and this is what we see as a result of our data transformation:TweetsRawDataNow we can easily use the reporting capabilities of StyleVision to group this data by language within each topic and count the number of posts in each language. We can then report this data in the form of pie charts, which produces the following interesting results:TweetsByLanguageObviously, this data is highly dependent on the date of execution and time of day, as well as the particular announcements happening about these products, so the numbers will fluctuate quite a bit, but it can be used as a nice monitoring for seeing different language-specific trends. And once this has been set up, the report can be refreshed easily with the click of a button to get a snapshot at that point in time. For more long-term analysis it would of course be necessary to modify the mapping a bit to query more than 100 recent tweets.In this article we have used Twitter’s Search API as one example data source and only looked at language as one unique data point, but there are many more interesting sources of data available online today, and this approach can be used on all of them in a similar fashion.If you want to experiment with other data sources and other kinds of information that you want to extract, we invite you to try for yourself. A free 30-day evaluation version of MapForce is available, and there are no limits on how you can use the other features of Altova’s data mapping and conversion tool for data processing tasks that go beyond analyzing social media trends…

Tags: , , , ,