Public:BWmeta service/Transformers
From YaddaWiki
(19 intermediate revisions not shown) | |||
Line 15: | Line 15: | ||
|- | |- | ||
| 2.0.0 || || || [[Image:Tick.png]] || | | 2.0.0 || || || [[Image:Tick.png]] || | ||
+ | |- | ||
+ | | 2.1.0 || || || [[Image:Tick.png]] || | ||
|} | |} | ||
Line 39: | Line 41: | ||
* <code>IMetadataFormatConverter</code> converts serialized metadata from one format to another one | * <code>IMetadataFormatConverter</code> converts serialized metadata from one format to another one | ||
- | + | See detailed description of transformers interface below. | |
+ | |||
+ | === Existing transformers in YADDA === | ||
+ | |||
+ | Existing transformers are located in the <code>pl.edu.icm.yadda.bwmeta.transformers</code> package. | ||
+ | |||
+ | Readers: | ||
+ | {| class="bluetable" | ||
+ | ! Class name !! Format !! Model | ||
+ | |- | ||
+ | | Bwmeta1_0ToDLTransformer || BWmeta 1.0 || DL | ||
+ | |- | ||
+ | | Bwmeta1_0ToRepoTransformer || BWmeta 1.0 || Repo | ||
+ | |- | ||
+ | | Bwmeta1_2ToYTransformer || BWmeta 1.2 || Y | ||
+ | |- | ||
+ | | Bwmeta2_0ToYTransformer || BWmeta 2.0 || Y | ||
+ | |- | ||
+ | | Bwmeta2_1ToYTransformer || BWmeta 2.1 || Y | ||
+ | |} | ||
+ | |||
+ | Writers: | ||
+ | {| class="bluetable" | ||
+ | ! Class name !! Model !! Format | ||
+ | |- | ||
+ | | DLToBwmeta1_0Transformer || DL || BWmeta 1.0 | ||
+ | |- | ||
+ | | RepoToBwmeta1_0Transformer || Repo || BWmeta 1.0 | ||
+ | |- | ||
+ | | YToBwmeta1_2Transformer || Y || BWmeta 1.2 | ||
+ | |- | ||
+ | | YToBwmeta2_0Transformer || Y || BWmeta 2.0 | ||
+ | |- | ||
+ | | YToBwmeta2_1Transformer || Y || BWmeta 2.1 | ||
+ | |} | ||
+ | |||
+ | Model converters: | ||
+ | {| class="bluetable" | ||
+ | ! Class name !! Source model !! Target model | ||
+ | |- | ||
+ | | DLToYTransformer || DL || Y | ||
+ | |- | ||
+ | | YToDLTransformer || Y || DL | ||
+ | |} | ||
+ | |||
+ | === MetadataTransformerFactory === | ||
+ | |||
+ | <code>MetadataTransformerFactory</code> maintains a list of available "atomic" transformers | ||
+ | and is able to combine the atomic transformers into more complex ones. | ||
+ | |||
+ | === MetadataTransformerFactory in YADDA === | ||
+ | |||
+ | <code>pl.edu.icm.yadda.bwmeta.transformers.BwmetaTransformers</code> contains a static instance of MetadataTransformerFactory (BTF), which can be used to obtain a transformer (reader, writer, model or format converter) between given formats and/or models. All new transformers should be added to the factory. | ||
+ | |||
+ | '''Repo transformers (reader and writer) are currently not included in BTF.''' However, there is a simple way to add new transformers to an existing factory without changing it: all you have to do is build a new factory from an existing one and additional transformers using MetadataTransformerFactory constructor. For example, to build a factory including BTF and Repo readers and writers: | ||
+ | |||
+ | <code> | ||
+ | IMetadataTransformerFactory myFactory = new MetadataTransformerFactory( | ||
+ | (MetadataTransformerFactory) BwmetaTransformers.BTF, | ||
+ | Arrays.<IMetadataReader<?>>asList(new Bwmeta1_0ToRepoTransformer()), | ||
+ | Arrays.<IMetadataWriter<?>>asList(new RepoToBwmeta1_0Transformer()), | ||
+ | Arrays.<IMetadataModelConverter<?, ?>>asList(), | ||
+ | Arrays.<IMetadataFormatConverter>asList()); | ||
+ | </code> | ||
+ | |||
+ | For factory usage, see detailed description of MetadataTransformerFactory below. | ||
+ | |||
+ | === Hints === | ||
+ | |||
+ | Hints are objects that can be passed to transformers to change their internal behaviour. | ||
+ | |||
+ | Hints can be either passed directly to transformers methods (in the form of <code>Object... hints</code> methods argument, see transformers interface below), or set in the factory as "global hints" (through factory constructor, see factory interface below). Global hints set in factory are passed automatically to all transformation methods in transformers obtained from the factory. | ||
+ | |||
+ | Hints can be either simple flags declared as static String instances in <code>pl.edu.icm.yadda.bwmeta.transformers.BwmetaTransformers</code>, or objects needed by transformers. | ||
+ | |||
+ | Currently there are two supported flag hints: | ||
+ | |||
+ | * FAST_TRANSFORMER - no validation | ||
+ | Normally writers try to validate the input. With this hint passed, they will skip the validation stage to make transformation faster. | ||
+ | |||
+ | * NO_BWMETA_ROOT - when possible, skip "bwmeta" root in output XML documents | ||
+ | This hint applies to writers only. Normally, regardless of the amount of model objects passed to a writer, they are all surrounded by a <bwmeta></bwmeta> root element. With this hint passed, the <bwmeta></bwmeta> root element will not be included if possible, that is, if there is only one model object passed to the writer. | ||
+ | |||
+ | There are also two supported object hints: | ||
+ | |||
+ | * Ancestor - element Ancestor data | ||
+ | * CategoryCodeMapper - mapping between category extId and (classificationId, code) | ||
+ | |||
+ | Please note, that there is a serious category code mapping issue whenever conversion between Bwmeta 1/DL model/repo model and Bwmeta2/YModel comes to play. Refer to [[Category service/Code mapping]] to understand and resolve the potential problem. This problem shows as categories 'missed' during conversion. | ||
+ | |||
+ | === Transformers interface === | ||
+ | |||
+ | Classes listed in Unified interface section provide following methods: | ||
<code>IMetadataReader<T></code> (T is a model class) | <code>IMetadataReader<T></code> (T is a model class) | ||
- | * <code>MetadataFormat getSourceFormat()</code> -- | + | * <code>MetadataFormat getSourceFormat()</code> -- returns the source format |
* <code>MetadataModel<T> getTargetModel()</code> -- returns the target model | * <code>MetadataModel<T> getTargetModel()</code> -- returns the target model | ||
* <code>List<T> read(String string, Object... hints) throws TransformationException</code> -- deserializes metadata from string; returns a list of model objects. | * <code>List<T> read(String string, Object... hints) throws TransformationException</code> -- deserializes metadata from string; returns a list of model objects. | ||
Line 62: | Line 156: | ||
* <code>MetadataFormat getTargetFormat()</code> -- returns the target format | * <code>MetadataFormat getTargetFormat()</code> -- returns the target format | ||
* <code>String convert(String source, Object... hints) throws TransformationException</code> -- converts serialized metadata given in <code>source</code> string; returns a string containing metadata in target format | * <code>String convert(String source, Object... hints) throws TransformationException</code> -- converts serialized metadata given in <code>source</code> string; returns a string containing metadata in target format | ||
- | * <code>void convert(Reader from, Writer to, Object... hints) throws TransformationException</code> -- converts serialized metadata read from a stream <code>from</code>; metadata in target format | + | * <code>void convert(Reader from, Writer to, Object... hints) throws TransformationException</code> -- converts serialized metadata read from a stream <code>from</code>; metadata in target format is written into a <code>to</code> stream. |
- | === | + | === MetadataTransformerFactory interface === |
- | + | Public constructors provided by <code>MetadataTransformerFactory</code>: | |
- | + | * constructor that builds a MetadataTransformerFactory from given lists of atomic transformers: | |
- | * | + | MetadataTransformerFactory(List<IMetadataReaders>, List<IMetadataWriters>, List<IMetadataModelConverters>, |
- | + | List<IMetadataFormatConverters>) | |
- | + | ||
- | + | ||
- | + | * constructor that builds a MetadataTransformerFactory from given lists of atomic transformers and global hints: | |
- | * | + | MetadataTransformerFactory(List<IMetadataReader>, List<IMetadataWriter>, List<IMetadataModelConverter>, |
- | + | List<IMetadataFormatConverter>, Object[]) | |
- | + | ||
- | + | ||
- | + | * constructor that creates a new MetadataTransformerFactory from an existing one, which contains joined lists of atomic transformers: | |
- | * | + | MetadataTransformerFactory(MetadataTransformerFactory, List<IMetadataReaders>, List<IMetadataWriters>, |
- | + | List<IMetadataModelConverters>, List<IMetadataFormatConverters>) | |
- | + | * contructor that creates a new MetadataTransformerFactory from an existing one and global hints | |
+ | MetadataTransformerFactory(MetadataTransformerFactory, Object[]) | ||
- | + | * constructor that build a new MetadataTransformerFactory from an existing one, list of transformers and global hints: | |
- | and | + | MetadataTransformerFactory(MetadataTransformerFactory, List<IMetadataReader>, List<IMetadataWriter>, |
- | + | List<IMetadataModelConverter>, List<IMetadataFormatConverter>, Object[]) { | |
- | + | ||
- | + | ||
- | + | ||
- | Public methods of <code>MetadataTransformerFactory</code>, that | + | Public methods of <code>MetadataTransformerFactory</code>, that returns transformer built from atomic transformers given in object constructor: |
* <T> IMetadataReader<T> getReader(MetadataModel<T>, MetadataFormat) | * <T> IMetadataReader<T> getReader(MetadataModel<T>, MetadataFormat) | ||
* <T> IMetadataWriter<T> getWriter(MetadataModel<T>, MetadataFormat) | * <T> IMetadataWriter<T> getWriter(MetadataModel<T>, MetadataFormat) | ||
Line 109: | Line 197: | ||
* IMetadataModelConverter compose(IMetadataWriter, IMetadataReader) | * IMetadataModelConverter compose(IMetadataWriter, IMetadataReader) | ||
- | === | + | === Old transformers usage in the YADDA source === |
+ | |||
+ | BwmetaReader105: | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta1_0ToDLTransformer.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaSerializer105.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105Test.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/DLMObjectsCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java | ||
+ | * ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/AbstractDLMImporter.java | ||
+ | * ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/DLMMultithreadImporter.java | ||
+ | * ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java | ||
+ | * ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java | ||
+ | |||
+ | BwmetaReader120: | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/dublincore/BWMeta120ToDCConverter.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaReader120.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta1_2ToYTransformer.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta120SerializationTest.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHPackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/medline/MedlinePackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCPackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerPackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java | ||
+ | * ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java | ||
+ | * ./yadda-analysis-impl/trunk/src/test/java/pl/edu/icm/yadda/analysis/bibref/YToBibEntryTransformerTest.java | ||
+ | * ./yadda-analysis/trunk/src/test/java/pl/edu/icm/yadda/analysis/bibref/YToBibEntryTransformerTest.java | ||
+ | * ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/parsing/AbstractCitationParserTest.java | ||
+ | * ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipFeederTest.java | ||
+ | * ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementBuilderExportTest.java | ||
+ | * ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementsBuilderTest.java | ||
+ | |||
+ | BwmetaReader2: | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaReader2.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta2_0ToYTransformer.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta20SerializationTest.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/export/PackCreatorBasedExporter.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/commons/impl/PackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerNewMetadataSource.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java | ||
+ | * ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/persons/BasicAbstractTest.java | ||
+ | * ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementBuilderExportTest.java | ||
+ | |||
+ | BwmetaWriter105: | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/DLToBwmeta1_0Transformer.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaSerializer105.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaWriter105.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105Test.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/serialization/BwmetaExporter.java | ||
+ | * ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java | ||
+ | * ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/exporter/SimpleBwmetaExporter.java | ||
+ | * ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/util/FlatCategoryParser.java | ||
+ | |||
+ | BwmetaWriter120: | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaWriter120.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToBwmeta1_2Transformer.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta120SerializationTest.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHPackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/medline/MedlinePackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCPackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerPackCreator.java | ||
+ | * ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java | ||
+ | * ./bwmeta-import/trunk/test/integration/pl/edu/icm/yadda/imports/springer/SpringerNewMetadataSourceTest.java | ||
+ | * ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java | ||
+ | * ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java | ||
+ | |||
+ | BwmetaWriter20: | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaWriter20.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToBwmeta2_0Transformer.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta20SerializationTest.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/export/PackCreatorBasedExporter.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java | ||
+ | |||
+ | BwmetaConverterDLToNew: | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNew.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/DLToYTransformer.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java | ||
+ | * ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java | ||
+ | * ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java | ||
- | + | BwmetaConverterNewToDL: | |
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDL.java | ||
+ | * ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToDLTransformer.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java | ||
+ | * ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHMetadataSource.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCMetadataSource.java | ||
+ | * ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java | ||
+ | * ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java |
Latest revision as of 15:18, 7 March 2013
Service details | |
---|---|
Name | BWmeta |
Code location (relative to SVN root) | projects/dir/bwmeta-core |
Javadoc | |
Contact person | Jakub Jurkiewicz |
Contents |
Background and motivation
As of 2010Q4, three versions of BWmeta format and at least three Java models of BWmeta are used in YADDA. See here for details. Serialization and deserialization is implemented for the following pairs:
Repo | DL | Y | … | |
---|---|---|---|---|
1.0.5 | ||||
1.2.0 | ||||
2.0.0 | ||||
2.1.0 |
There are also converters between selected models:
- Y ↔ DL
Serialization of Y model to BWmeta 1.0.5 is therefore possible through the following chain of transformations:
- Y model → DL model → BWmeta 1.0.5
Conversion from DL model to Repo model is possible through:
- DL model → BWmeta 1.0.5 → Repo model
Unfortunately, many mutually-incompatible transformation interfaces are present. It is difficult to understand which transformations are possible, how to combine them, which classes should be used for that end and how to instantiate the classes.
A unified interface to all the transformations would be very helpful.
Unified interface
Four different transformer types are present: readers, writers and model converters and format converters. Each type has a corresponding interface:
-
IMetadataReader<T>
deserializes an object of typeT
from aString
or aReader
-
IMetadataWriter<T>
serializes an object of typeT
to aString
or aWriter
-
IMetadataModelConverter<S, T>
converts an object of typeS
to an object of typeT
-
IMetadataFormatConverter
converts serialized metadata from one format to another one
See detailed description of transformers interface below.
Existing transformers in YADDA
Existing transformers are located in the pl.edu.icm.yadda.bwmeta.transformers
package.
Readers:
Class name | Format | Model |
---|---|---|
Bwmeta1_0ToDLTransformer | BWmeta 1.0 | DL |
Bwmeta1_0ToRepoTransformer | BWmeta 1.0 | Repo |
Bwmeta1_2ToYTransformer | BWmeta 1.2 | Y |
Bwmeta2_0ToYTransformer | BWmeta 2.0 | Y |
Bwmeta2_1ToYTransformer | BWmeta 2.1 | Y |
Writers:
Class name | Model | Format |
---|---|---|
DLToBwmeta1_0Transformer | DL | BWmeta 1.0 |
RepoToBwmeta1_0Transformer | Repo | BWmeta 1.0 |
YToBwmeta1_2Transformer | Y | BWmeta 1.2 |
YToBwmeta2_0Transformer | Y | BWmeta 2.0 |
YToBwmeta2_1Transformer | Y | BWmeta 2.1 |
Model converters:
Class name | Source model | Target model |
---|---|---|
DLToYTransformer | DL | Y |
YToDLTransformer | Y | DL |
MetadataTransformerFactory
MetadataTransformerFactory
maintains a list of available "atomic" transformers
and is able to combine the atomic transformers into more complex ones.
MetadataTransformerFactory in YADDA
pl.edu.icm.yadda.bwmeta.transformers.BwmetaTransformers
contains a static instance of MetadataTransformerFactory (BTF), which can be used to obtain a transformer (reader, writer, model or format converter) between given formats and/or models. All new transformers should be added to the factory.
Repo transformers (reader and writer) are currently not included in BTF. However, there is a simple way to add new transformers to an existing factory without changing it: all you have to do is build a new factory from an existing one and additional transformers using MetadataTransformerFactory constructor. For example, to build a factory including BTF and Repo readers and writers:
IMetadataTransformerFactory myFactory = new MetadataTransformerFactory( (MetadataTransformerFactory) BwmetaTransformers.BTF, Arrays.<IMetadataReader<?>>asList(new Bwmeta1_0ToRepoTransformer()), Arrays.<IMetadataWriter<?>>asList(new RepoToBwmeta1_0Transformer()), Arrays.<IMetadataModelConverter<?, ?>>asList(), Arrays.<IMetadataFormatConverter>asList());
For factory usage, see detailed description of MetadataTransformerFactory below.
Hints
Hints are objects that can be passed to transformers to change their internal behaviour.
Hints can be either passed directly to transformers methods (in the form of Object... hints
methods argument, see transformers interface below), or set in the factory as "global hints" (through factory constructor, see factory interface below). Global hints set in factory are passed automatically to all transformation methods in transformers obtained from the factory.
Hints can be either simple flags declared as static String instances in pl.edu.icm.yadda.bwmeta.transformers.BwmetaTransformers
, or objects needed by transformers.
Currently there are two supported flag hints:
- FAST_TRANSFORMER - no validation
Normally writers try to validate the input. With this hint passed, they will skip the validation stage to make transformation faster.
- NO_BWMETA_ROOT - when possible, skip "bwmeta" root in output XML documents
This hint applies to writers only. Normally, regardless of the amount of model objects passed to a writer, they are all surrounded by a <bwmeta></bwmeta> root element. With this hint passed, the <bwmeta></bwmeta> root element will not be included if possible, that is, if there is only one model object passed to the writer.
There are also two supported object hints:
- Ancestor - element Ancestor data
- CategoryCodeMapper - mapping between category extId and (classificationId, code)
Please note, that there is a serious category code mapping issue whenever conversion between Bwmeta 1/DL model/repo model and Bwmeta2/YModel comes to play. Refer to Category service/Code mapping to understand and resolve the potential problem. This problem shows as categories 'missed' during conversion.
Transformers interface
Classes listed in Unified interface section provide following methods:
IMetadataReader<T>
(T is a model class)
-
MetadataFormat getSourceFormat()
-- returns the source format -
MetadataModel<T> getTargetModel()
-- returns the target model -
List<T> read(String string, Object... hints) throws TransformationException
-- deserializes metadata from string; returns a list of model objects. -
List<T> read(Reader reader, Object... hints) throws TransformationException
-- deserializes metadata fromreader
stream; returns a list of model objects.
IMetadataWriter<T>
-
MetadataModel<T> getSourceModel()
-- returns the source model -
MetadataFormat getTargetFormat()
-- returns the target format -
String write(List<T> objects, Object... hints) throws TransformationException
-- serializes metadata model objects given inobjects
list; returns a string contains all metadata -
void write(Writer writer, List<T> objects, Object... hints) throws TransformationException
-- serializes metadata model objects given inobjects
list; the result is written intowriter
stream.
IMetadataModelConverter<S, T>
-
MetadataModel<S> getSourceModel()
-- returns the source model -
MetadataModel<T>
-- returns the target model -
T convert(S source, Object... hints) throws TransformationException;
-- converts a metadata object of type S into objects of type T
IMetadataFormatConverter
-
MetadataFormat getSourceFormat()
-- returns the source format -
MetadataFormat getTargetFormat()
-- returns the target format -
String convert(String source, Object... hints) throws TransformationException
-- converts serialized metadata given insource
string; returns a string containing metadata in target format -
void convert(Reader from, Writer to, Object... hints) throws TransformationException
-- converts serialized metadata read from a streamfrom
; metadata in target format is written into ato
stream.
MetadataTransformerFactory interface
Public constructors provided by MetadataTransformerFactory
:
- constructor that builds a MetadataTransformerFactory from given lists of atomic transformers:
MetadataTransformerFactory(List<IMetadataReaders>, List<IMetadataWriters>, List<IMetadataModelConverters>, List<IMetadataFormatConverters>)
- constructor that builds a MetadataTransformerFactory from given lists of atomic transformers and global hints:
MetadataTransformerFactory(List<IMetadataReader>, List<IMetadataWriter>, List<IMetadataModelConverter>, List<IMetadataFormatConverter>, Object[])
- constructor that creates a new MetadataTransformerFactory from an existing one, which contains joined lists of atomic transformers:
MetadataTransformerFactory(MetadataTransformerFactory, List<IMetadataReaders>, List<IMetadataWriters>, List<IMetadataModelConverters>, List<IMetadataFormatConverters>)
- contructor that creates a new MetadataTransformerFactory from an existing one and global hints
MetadataTransformerFactory(MetadataTransformerFactory, Object[])
- constructor that build a new MetadataTransformerFactory from an existing one, list of transformers and global hints:
MetadataTransformerFactory(MetadataTransformerFactory, List<IMetadataReader>, List<IMetadataWriter>, List<IMetadataModelConverter>, List<IMetadataFormatConverter>, Object[]) {
Public methods of MetadataTransformerFactory
, that returns transformer built from atomic transformers given in object constructor:
- <T> IMetadataReader<T> getReader(MetadataModel<T>, MetadataFormat)
- <T> IMetadataWriter<T> getWriter(MetadataModel<T>, MetadataFormat)
- <S, T> IMetadataModelConverter<S, T> getModelConverter(MetadataModel<S>, MetadataModel<T>)
- IMetadataFormatConverter getFormatConverter(MetadataFormat, MetadataFormat)
MetadataTransformerFactory
also contains a set of static methods compose()
, that return a complex transformer built from two given transformers. These methods are not part of IMetadataTransformerFactory interface.
- IMetadataReader compose(IMetadataFormatConverter, IMetadataReader)
- IMetadataReader compose(IMetadataReader, IMetadataModelConverter)
- IMetadataWriter compose(IMetadataWriter, IMetadataFormatConverter)
- IMetadataWriter compose(IMetadataModelConverter, IMetadataWriter)
- IMetadataFormatConverter compose(IMetadataFormatConverter, IMetadataFormatConverter)
- IMetadataFormatConverter compose(IMetadataReader, IMetadataWriter)
- IMetadataModelConverter compose(IMetadataModelConverter, IMetadataModelConverter)
- IMetadataModelConverter compose(IMetadataWriter, IMetadataReader)
Old transformers usage in the YADDA source
BwmetaReader105:
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta1_0ToDLTransformer.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaSerializer105.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105Test.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/DLMObjectsCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
- ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/AbstractDLMImporter.java
- ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/importer/DLMMultithreadImporter.java
- ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java
- ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java
BwmetaReader120:
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/dublincore/BWMeta120ToDCConverter.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaReader120.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta1_2ToYTransformer.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta120SerializationTest.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHPackCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/medline/MedlinePackCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCPackCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerPackCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
- ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java
- ./yadda-analysis-impl/trunk/src/test/java/pl/edu/icm/yadda/analysis/bibref/YToBibEntryTransformerTest.java
- ./yadda-analysis/trunk/src/test/java/pl/edu/icm/yadda/analysis/bibref/YToBibEntryTransformerTest.java
- ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/parsing/AbstractCitationParserTest.java
- ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipFeederTest.java
- ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementBuilderExportTest.java
- ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementsBuilderTest.java
BwmetaReader2:
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaReader2.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/Bwmeta2_0ToYTransformer.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta20SerializationTest.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/export/PackCreatorBasedExporter.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/commons/impl/PackCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerNewMetadataSource.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
- ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/persons/BasicAbstractTest.java
- ./yadda-content/trunk/src/test/java/pl/edu/icm/yadda/tools/relations/RelationshipStatementBuilderExportTest.java
BwmetaWriter105:
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/DLToBwmeta1_0Transformer.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaSerializer105.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/desklight/serialization/BwmetaWriter105.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/desklight/serialization/BwmetaReader105Test.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/desklight/model/serialization/BwmetaExporter.java
- ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java
- ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/exporter/SimpleBwmetaExporter.java
- ./DeskLight/trunk/src/main/java/pl/edu/icm/yadda/desklight/util/FlatCategoryParser.java
BwmetaWriter120:
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaWriter120.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToBwmeta1_2Transformer.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta120SerializationTest.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHPackCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/medline/MedlinePackCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCPackCreator.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/springer/SpringerPackCreator.java
- ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java
- ./bwmeta-import/trunk/test/integration/pl/edu/icm/yadda/imports/springer/SpringerNewMetadataSourceTest.java
- ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java
- ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java
BwmetaWriter20:
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/serialization/BwmetaWriter20.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToBwmeta2_0Transformer.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/serialization/Bwmeta20SerializationTest.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/export/PackCreatorBasedExporter.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
BwmetaConverterDLToNew:
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNew.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/DLToYTransformer.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLTo20Test.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterDLToNewTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
- ./yadda-content/trunk/src/main/java/pl/edu/icm/yadda/tools/relations/RelationshipStorageFeeder.java
- ./yadda-oaiserver/trunk/src/main/java/pl/edu/icm/yadda/oaiserver/catalog/BwmetaCatalog.java
BwmetaConverterNewToDL:
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDL.java
- ./bwmeta-core/trunk/src/main/java/pl/edu/icm/yadda/bwmeta/transformers/YToDLTransformer.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/Bwmeta20ConverterTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterCategorizationNewToDLTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterNewToDLTest.java
- ./bwmeta-core/trunk/src/test/java/pl/edu/icm/yadda/bwmeta/converters/BwmetaConverterTest.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/cejsh/CEJSHMetadataSource.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/imports/psjc/PSJCMetadataSource.java
- ./bwmeta-import/trunk/src/main/java/pl/edu/icm/yadda/tools/BasicPackImporter.java
- ./bwmeta-import/trunk/src/test/java/pl/edu/icm/yadda/imports/springer/SpringerParserTest.java