--- nfo/perl/libs/Data/Transfer/Sync/Core.pm 2003/01/19 01:23:04 1.1 +++ nfo/perl/libs/Data/Transfer/Sync/Core.pm 2003/01/19 02:05:42 1.2 @@ -1,4 +1,4 @@ -## $Id: Core.pm,v 1.1 2003/01/19 01:23:04 joko Exp $ +## $Id: Core.pm,v 1.2 2003/01/19 02:05:42 joko Exp $ ## ## Copyright (c) 2002 Andreas Motl ## @@ -6,6 +6,9 @@ ## ## ---------------------------------------------------------------------------------------- ## $Log: Core.pm,v $ +## Revision 1.2 2003/01/19 02:05:42 joko +## - removed pod-documentation: now in Data/Transfer/Sync.pod +## ## Revision 1.1 2003/01/19 01:23:04 joko ## + new from Data/Transfer/Sync.pm ## @@ -1204,145 +1207,4 @@ $self->{meta}->{$descent}->{storage}->eraseAll($node); } - -=pod - - -=head1 DESCRIPTION - -Data::Transfer::Sync is a module providing a generic synchronization process -across arbitrary/multiple storages based on a ident/checksum mechanism. -It sits on top of Data::Storage. - - -=head1 REQUIREMENTS - - For full functionality: - Data::Storage - Data::Transform - Data::Compare - ... and all their dependencies - - -=head1 AUTHORS / COPYRIGHT - -The Data::Storage module is Copyright (c) 2002 Andreas Motl. -All rights reserved. - -You may distribute it under the terms of either the GNU General Public -License or the Artistic License, as specified in the Perl README file. - - -=head1 SUPPORT / WARRANTY - -Data::Storage is free software. IT COMES WITHOUT WARRANTY OF ANY KIND. - - - -=head1 BUGS - - When in "import" mode for windows file - DBD::AutoCSV may hang. - Hint: Maybe the source node contains an ident-, but no checksum-column? - - -=head1 USER LEVEL ERRORS - -=head4 Mapping - - - - - - - - - - - - - - - - - - - - - - - - - - - - - info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 )critical: BizWorks::Process::Setup->startSync: Can't access mapping for node "Currency" - please check BizWorks::ResourceMapping. - - - - - - - - - - - - - - - - - - - - - - - - - - - - You have to create a sub for each node used in synchronization inside named Perl module. The name of this sub _must_ match - the name of the node you want to sync. This sub holds mapping metadata to give the engine hints about how - to access the otherwise generic nodes. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -=head4 DBD::AutoCSV's rulebase - - - - - - - - - - - - - - - - - - - - - - - - - - - - - info: BizWorks::Process::Setup->syncResource( source_node Currency mode PULL erase 0 import 0 ) - info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv - - Execution ERROR: Error while scanning: Missing first row or scanrule not applied. at C:/home/amo/develop/netfrag.org/nfo/perl/libs/DBD/CSV.p - m line 165, line 1. - called from C:/home/amo/develop/netfrag.org/nfo/perl/libs/Data/Storage/Handler/DBI.pm at 123. - - DBI-Error: DBD::AutoCSV::st fetchrow_hashref failed: Attempt to fetch row from a Non-SELECT statement - notice: Data::Transfer::Sync->syncNodes: No nodes to synchronize. - - - - - - - - - - - - - - - - - - - - - - - - - - - - DBD::AutoCSV contains a rulebase which is spooled down while attempting to guess the style of the csv file regarding - parameters like newline (eol), column-seperation-character (sep_char), quoting character (quote_char). - If this spool runs out of entries and no style could be resolved, DBD::CSV dies causing this "Execution ERROR" which - results in a "DBI-Error" afterwards. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -=head4 Check structure of source node - - - - - - - - - - - - - - - - - - - - - - - - - - - - - info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv - critical: Data::Transfer::Sync->syncNodes: Can not synchronize: No ident found in source node, maybe try to "import" this node first. - - - - - - - - - - - - - - - - - - - - - - - - - - - - If lowlevel detection succeeds, but no other required informations are found, this message is issued. - "Other informations" might be: - - column-header-row completely missing - - ident column is empty - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -=head4 Modify structure of source node - - - - - - - - - - - - - - - - - - - - - - - - - - - - - info: Data::Transfer::Sync->syncNodes: source=L/Currency <- target=R/currencies.csv - info: Data::Transfer::Sync->_prepareNode_MetaProperties( descent source ) - warning: Data::Transfer::Sync->_prepareNode_MetaProperties: node is lacking meta properties - will try to alter... - SQL ERROR: Command 'ALTER' not recognized or not supported! - - SQL ERROR: Command 'ALTER' not recognized or not supported! - - - - - - - - - - - - - - - - - - - - - - - - - - - - The Engine found a node which structure does not match the required. It tries to alter this automatically - only when doing "import" - - but the DBD driver (in this case DBD::CSV) gets in the way croaking not to be able to do this. - This could also appear if your database connection has insufficient rights to modify the database structure. - DBD::CSV croaks because it doesn't implement the ALTER command, so please edit your columns manually. - Hint: Add columns with the names of your "ident" and "checksum" property specifications. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -=head4 Load source node by ident - - - - - - - - - - - - - - - - - - - - - - - - - - - - - info: Data::Transfer::Sync->_prepareNode_DummyIdent( descent source ) - pcritical: Data::Transfer::Sync->_modifyNode failed: "source" node is empty. - - - - - - - - - - - - - - - - - - - - - - - - - - - - The source node could not be loaded. Maybe the ident is missing. Please check manually. - Hint: Like above, the ident and/or checksum columns may be missing.... - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -=head1 TODO - - - sub _resolveIdentProvider - - wrap _doModifySource and _doTransferTarget around a core function which can change virtually any type of node - - split this module up into Sync.pm, Sync/Core.pm, Sync/Compare.pm and Sync/Compare/Checksum.pm - - introduce _compareNodes as a core method and wrap it around methods in Sync/Compare/Checksum.pm - - introduce Sync/Compare/MyComparisonImplementation.pm - - some generic deferring method - e.g. "$self->defer(action)" - to be able to accumulate a bunch of actions for later processing - - this implies everything done is _really_ split up into generic actions - how else would we defer them??? - - example uses: - - fetch whole checksum list from node - - remember source ident retransmits - - remember: this is convenient - and maybe / of course faster - but we'll loose "per-node-atomic" operations - - feature: mechanism to implicit inject checksum property to nodes (alter table / modify schema) - - expand statistics / keep track of: - - touched/untouched nodes - - full sync - - just do a push and a pull for now but use stats for touched nodes in between to speed up things - - introduce some new metadata flags for a synchronization partner which is (e.g.) of "source" or "target": - - isNewNodePropagator - - isWriteProtected - - -=cut - 1;