Public:BWmeta service/Transformers

From YaddaWiki

(Difference between revisions)
Jump to: navigation, search
(→Old transformers in the YADDA source)
Line 98: Line 98:
See detailed description of MetadataTransformerFactory below.
See detailed description of MetadataTransformerFactory below.
 +
 +
=== Hints ===
 +
 +
Transformers interface (see below) lets to pass additional information to all transformation methods (in the form of <code>Object... hints</code> methods argument). This can be used to change internal transformers behaviour according to our needs.
 +
 +
Hints are declared as static String instances in <code>pl.edu.icm.yadda.bwmeta.transformers.BwmetaTransformers</code>.
 +
 +
Currently there are two supported hints:
 +
 +
* FAST_TRANSFORMER - no validation
 +
Normally readers and writer try to validate the input. With this hint passed, they will skip the validation stage to make transformation faster.
 +
 +
* NO_BWMETA_ROOT - when possible, skip "bwmeta" root in output XML documents
 +
This hint applies to writers only. Normally, regardless of the amount of model objects passed to a writer, they are all surrounded by a <bwmeta></bwmeta> root element. With this hint passed, the <bwmeta></bwmeta> root element will not be included if possible, that is, if there is only one model object passed to the writer.
=== Transformers interface ===  
=== Transformers interface ===  

Revision as of 12:33, 17 January 2011

Service details
NameBWmeta
Code location
(relative to SVN root)		projects/dir/bwmeta-core
Javadoc
Contact personJakub Jurkiewicz

Contents

Background and motivation

As of 2010Q4, three versions of BWmeta format and at least three Java models of BWmeta are used in YADDA. See here for details. Serialization and deserialization is implemented for the following pairs:

Repo DL Y …
1.0.5 Image:Tick.png Image:Tick.png
1.2.0 Image:Tick.png
2.0.0 Image:Tick.png
2.1.0 Image:Tick.png

There are also converters between selected models:

  • Y ↔ DL

Serialization of Y model to BWmeta 1.0.5 is therefore possible through the following chain of transformations:

  • Y model → DL model → BWmeta 1.0.5

Conversion from DL model to Repo model is possible through:

  • DL model → BWmeta 1.0.5 → Repo model

Unfortunately, many mutually-incompatible transformation interfaces are present. It is difficult to understand which transformations are possible, how to combine them, which classes should be used for that end and how to instantiate the classes.

A unified interface to all the transformations would be very helpful.

Unified interface

Four different transformer types are present: readers, writers and model converters and format converters. Each type has a corresponding interface:

  • IMetadataReader<T> deserializes an object of type T from a String or a Reader
  • IMetadataWriter<T> serializes an object of type T to a String or a Writer
  • IMetadataModelConverter<S, T> converts an object of type S to an object of type T
  • IMetadataFormatConverter converts serialized metadata from one format to another one

See detailed description of transformers interface below.

Existing transformers in YADDA

Existing transformers are located in the pl.edu.icm.yadda.bwmeta.transformers package.

Readers:

Class name Format Model
Bwmeta1_0ToDLTransformer BWmeta 1.0 DL
Bwmeta1_0ToRepoTransformer BWmeta 1.0 Repo
Bwmeta1_2ToYTransformer BWmeta 1.2 Y
Bwmeta2_0ToYTransformer BWmeta 2.0 Y
Bwmeta2_1ToYTransformer BWmeta 2.1 Y

Writers:

Class name Model Format
DLToBwmeta1_0Transformer DL BWmeta 1.0
RepoToBwmeta1_0Transformer Repo BWmeta 1.0
YToBwmeta1_2Transformer Y BWmeta 1.2
YToBwmeta2_0Transformer Y BWmeta 2.0
YToBwmeta2_1Transformer Y BWmeta 2.1

Model converters:

Class name Source model Target model
DLToYTransformer DL Y
YToDLTransformer Y DL

MetadataTransformerFactory

MetadataTransformerFactory maintains a list of available "atomic" transformers and is able to combine the atomic transformers into more complex ones.

MetadataTransformerFactory in YADDA

pl.edu.icm.yadda.bwmeta.transformers.BwmetaTransformers contains a static instance of MetadataTransformerFactory (BTF), which can be used to obtain a transformer (reader, writer, model or format converter) between given formats and/or models. All new transformers should be added to the factory.

Repo transformers (reader and writer) are currently not included in BTF.

See detailed description of MetadataTransformerFactory below.

Hints

Transformers interface (see below) lets to pass additional information to all transformation methods (in the form of Object... hints methods argument). This can be used to change internal transformers behaviour according to our needs.

Hints are declared as static String instances in pl.edu.icm.yadda.bwmeta.transformers.BwmetaTransformers.

Currently there are two supported hints:

  • FAST_TRANSFORMER - no validation

Normally readers and writer try to validate the input. With this hint passed, they will skip the validation stage to make transformation faster.

  • NO_BWMETA_ROOT - when possible, skip "bwmeta" root in output XML documents

This hint applies to writers only. Normally, regardless of the amount of model objects passed to a writer, they are all surrounded by a <bwmeta></bwmeta> root element. With this hint passed, the <bwmeta></bwmeta> root element will not be included if possible, that is, if there is only one model object passed to the writer.

Transformers interface

Classes listed in Unified interface section provide following methods:

IMetadataReader<T> (T is a model class)

  • MetadataFormat getSourceFormat() -- returns the source format
  • MetadataModel<T> getTargetModel() -- returns the target model
  • List<T> read(String string, Object... hints) throws TransformationException -- deserializes metadata from string; returns a list of model objects.
  • List<T> read(Reader reader, Object... hints) throws TransformationException -- deserializes metadata from reader stream; returns a list of model objects.

IMetadataWriter<T>

  • MetadataModel<T> getSourceModel() -- returns the source model
  • MetadataFormat getTargetFormat() -- returns the target format
  • String write(List<T> objects, Object... hints) throws TransformationException -- serializes metadata model objects given in objects list; returns a string contains all metadata
  • void write(Writer writer, List<T> objects, Object... hints) throws TransformationException -- serializes metadata model objects given in objects list; the result is written into writer stream.

IMetadataModelConverter<S, T>

  • MetadataModel<S> getSourceModel() -- returns the source model
  • MetadataModel<T> -- returns the target model
  • T convert(S source, Object... hints) throws TransformationException; -- converts a metadata object of type S into objects of type T

IMetadataFormatConverter

  • MetadataFormat getSourceFormat() -- returns the source format
  • MetadataFormat getTargetFormat() -- returns the target format
  • String convert(String source, Object... hints) throws TransformationException -- converts serialized metadata given in source string; returns a string containing metadata in target format
  • void convert(Reader from, Writer to, Object... hints) throws TransformationException -- converts serialized metadata read from a stream from; metadata in target format is written into a to stream.

MetadataTransformerFactory interface

Public constructors provided by MetadataTransformerFactory:

  • MetadataTransformerFactory(List<IMetadataReaders>, List<IMetadataWriters>, List<IMetadataModelConverters>, List<IMetadataFormatConverters>) -- builds a MetadataTransformerFactory from given lists of atomic transformers,
  • MetadataTransformerFactory(MetadataTransformerFactory, List<IMetadataReaders>, List<IMetadataWriters>, List<IMetadataModelConverters>, List<IMetadataFormatConverters>) -- creates a new MetadataTransformerFactory from an existing one, which contains joined lists of atomic transformers.

Public methods of MetadataTransformerFactory, that returns transformer built from atomic transformers given in object constructor:

  • <T> IMetadataReader<T> getReader(MetadataModel<T>, MetadataFormat)
  • <T> IMetadataWriter<T> getWriter(MetadataModel<T>, MetadataFormat)
  • <S, T> IMetadataModelConverter<S, T> getModelConverter(MetadataModel<S>, MetadataModel<T>)
  • IMetadataFormatConverter getFormatConverter(MetadataFormat, MetadataFormat)

MetadataTransformerFactory also contains a set of static methods compose(), that return a complex transformer built from two given transformers. These methods are not part of IMetadataTransformerFactory interface.

  • IMetadataReader compose(IMetadataFormatConverter, IMetadataReader)
  • IMetadataReader compose(IMetadataReader, IMetadataModelConverter)
  • IMetadataWriter compose(IMetadataWriter, IMetadataFormatConverter)
  • IMetadataWriter compose(IMetadataModelConverter, IMetadataWriter)
  • IMetadataFormatConverter compose(IMetadataFormatConverter, IMetadataFormatConverter)
  • IMetadataFormatConverter compose(IMetadataReader, IMetadataWriter)
  • IMetadataModelConverter compose(IMetadataModelConverter, IMetadataModelConverter)
  • IMetadataModelConverter compose(IMetadataWriter, IMetadataReader)

Old transformers usage in the YADDA source

BwmetaReader105:

  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta1_0ToDLTransformer.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaSerializer105.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105Test.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/DLMObjectsCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
  • ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/AbstractDLMImporter.java
  • ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/DLMMultithreadImporter.java
  • ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java
  • ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java

BwmetaReader120:

  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/dublincore/BWMeta120ToDCConverter.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaReader120.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta1_2ToYTransformer.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta120SerializationTest.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHPackCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/medline/MedlinePackCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCPackCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerPackCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
  • ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java
  • ./yadda-analysis-impl/trunk/src/test/java/pl/edu/icm/yadda/analysis/bibref/YToBibEntryTransformerTest.java
  • ./yadda-analysis/trunk/src/test/java/pl/edu/icm/yadda/analysis/bibref/YToBibEntryTransformerTest.java
  • ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/parsing/AbstractCitationParserTest.java
  • ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipFeederTest.java
  • ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementBuilderExportTest.java
  • ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementsBuilderTest.java

BwmetaReader2:

  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaReader2.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta2_0ToYTransformer.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta20SerializationTest.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/export/PackCreatorBasedExporter.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/commons/impl/PackCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerNewMetadataSource.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
  • ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/persons/BasicAbstractTest.java
  • ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementBuilderExportTest.java

BwmetaWriter105:

  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/DLToBwmeta1_0Transformer.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaSerializer105.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaWriter105.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105Test.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/serialization/BwmetaExporter.java
  • ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java
  • ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/exporter/SimpleBwmetaExporter.java
  • ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/util/FlatCategoryParser.java

BwmetaWriter120:

  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaWriter120.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToBwmeta1_2Transformer.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta120SerializationTest.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHPackCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/medline/MedlinePackCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCPackCreator.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerPackCreator.java
  • ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java
  • ./bwmeta-import/trunk/test/integration/pl/edu/icm/yadda/imports/springer/SpringerNewMetadataSourceTest.java
  • ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java
  • ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java

BwmetaWriter20:

  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaWriter20.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToBwmeta2_0Transformer.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta20SerializationTest.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/export/PackCreatorBasedExporter.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java

BwmetaConverterDLToNew:

  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNew.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/DLToYTransformer.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
  • ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java
  • ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java

BwmetaConverterNewToDL:

  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDL.java
  • ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToDLTransformer.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java
  • ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHMetadataSource.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCMetadataSource.java
  • ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
  • ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java
Retrieved from "https://wiki.yadda.icm.edu.pl/yadda/Public:BWmeta_service/Transformers"
Personal tools