Greg's InTouch Database Utilities and Filemaker Pro Templates

Greg Keraunen
This file last modified 12/28/05, 1:12 am
FILE:
	README

ARCHIVE:
	intouch_utils.sit

AUTHOR:
   Gregory Keraunen <gk@proliberty.com>
   Copyright Finantek Veinte, S.A., 2002, 2003, 2004

ARCHIVE CONTENTS:
	README
	addgroups.sh
	rmtabs.sh
	InTouch.fp3
	InTouchGroups.fp3

This package contains files for the purpose of exporting data from the
"InTouch" Macintosh ( http://prgrsoft.com ) addressbook to a form
suitable for import into Filemaker Pro.

I created these files and scripts so that my data could be shared between
Mac and Windows.

The problem arose because InTouch allows assigning records to any number of
groups (categories)
but it does not support export of these group relationships.

Another problem addressed is a bug in InTouch tab-delimited export method:
	any embedded tabs are NOT converted on export!
	(the correct behavior would be to convert tabs to spaces)

This means that if you try to import an exported file with embedded tabs it
will look like 3 or more fields per record instead of two and data will be
lost. rmtabs.sh fixes this
    + Embedded newlines appear to export from InTouch as vertical tabs;
    same as \r
    + Records without any data in subject field import notes field into
    subject field
       this is may be a bug but i like it

I chose to use Filemaker Pro's repeating fields to contain group membership
data.
The names of the groups themselves are imported to a separate, related
database that is used to populate
pop-up menus of group names in the primary database.

While this script and discussion is specific to the above mentioned
applications, it should also be useful
under the following conditions:
     the source database can export all records in a single tab-delimited
    file
     you have a set of subsets (groups) of the database in separate
    tab-delimited files
     the destination database can import tab-delimited files with
    REPEATIING fields, delimited by ascii 29 (35 octal)

This file describes:
     the usage of addgroups.sh to process a set of database files to
    create two new files
     the method of exporting data from InTouch
     workaround to InTouch's broken TAB file export
     the method of importing created files into Filemaker Pro

////////////// addgroups.sh

# USAGE:
#   addgroups.sh [-o output_file] [name of directory containing group files
# to process]

Example:
	$cd /intouch exports/
	$/path/to/addgroups.sh

INPUT FILES

    All Input files must be in DOS format

     one tab-delimited file named "All" with all records from source
    database (no group names are included)
     any number of additional group files, named the same as group name
    you want the records to belong to in the DESTINATION database

OUTPUT FILES

  DOS format

   "All.tab" - repeating field of group_name for each record added to
  original "All" file data
      
       <record_id>\t[<group_name>\035<group_name>]*\t<subject_field>\t<notes_field>\r\n

    "Groups.tab" - group names
       <group_name>\r\n

////////////// rmtabs.sh

# USAGE:
#   rmtabs.sh [-o output_file] [-t] [-s] [-v] [-h] [INPUT_FILE]

Example:
	$cd /intouch exports/
	$/path/to/rmtabs.sh -o All.rmtabs.tab All.tab

SETUP:
	This script requires tetex-1.0.7-15 for the command:
		readlink
	This command is not really necessary; you can modify the script if you
	don't have readlink on your system

//////////////  FILEMAKER TEMPLATE FILES

I have created the InTouch.fp3 and InTouchGroups.fp3 templates as a
somewhat improved version of InTouch

Although this application lacks many of the InTouch features (such as
dialing, reminder), I never used those.

Filemaker is more powerful but also slower than InTouch and I customized it
to give more control over group management:
	Group assignments are limited to 10 groups (only 5 displayed in Form View)
		this saves some screen space and is tidier than InTouch
	'Favorites' popup menu gives faster access to your favorite records
		I spent a lot of time scrolling through InTouch for my favorite records
		The Favorites popup menu is only updated if FavoriteItem fields change
		    FavoriteItem fields are only updated when user hits ASSIGN Group as
		    Favorites in Config screen
		    - if a user adds a record to the Favorite group, it is NOT updated to
		    create FavoriteItem content
		    - if a user edits the first line of a Favorite subject field, the
		    menu doesn't reflect the subject anymore
		    - the Favorites menu doesn't change to include newly added or
		    modified records until Config:ASSIGN

////////////// CONVERTING THE DATA
Now the fun part...

 InTouch: export each group as Tab-delimited
   Choose a filename that does NOT INCLUDE ANY high-ascii characters (such
   as Option-8; "bullet")
   It also should not contain any of the following characters: (see CROSS
   PLATFORM FILENAME ISSUES below)
     | \ " < > / ?
   The filename you choose will be used as the new group name

 BBEdit:
   open the exported files, then for each one:
	Save converted file as DOS format;
	IF you only plan on importing to Filemaker Pro, you can skip the next step
	since it has built-in
	character set mapping between Mac and Windows; otherwise:
		BBEdit:Tools: Convert to ascii
			this will automatically search and replace any high-ascii characters in
			data

 Copy all exported files InTouch files to linux machine where you can run
bash
 Copy addgroups.sh and rmtabs.sh to linux machine
 From linux:
    $cd <directory where you have put the exported InTouch files>
    $<path to addgroups.sh>
    $/path/to/rmtabs.sh -o All.rmtabs.tab All.tab

 Copy the output files All.rmtabs.tab and Groups.tab to somewhere that
Filemaker Pro can open them from

 FMPro:
   Open InTouchGroups.fp3; delete all records, if any found
   Import:
       set menu: "Show Tab Separated Text"
       open file: "Groups.tab"
       make sure default is checked: "Add new records"
       assign imported fields to: group_name
       Import

 FMPro:
   Open InTouch.fp3 template file;
   delete all records, if any found
   If you play around, adding any records to this file before doing your
   final import:
        go to Define Fields: Record ID; make sure the settings are:
        "next value =  1; increment by 1
   Import:
       set menu: "Show Tab Separated Text"
       open file: "All.rmtabs.tab"
       make sure default is checked: "Add new records"
       assign imported fields to: RecordID, Groups, Subject, Notes
       Import
       Yes (default) to dialog: "...perform auto-enter options while
       importing.."

//////////////  FILEMAKER IMPORT PROBLEMS / WORKAROUNDS
I use Filemaker Pro 3.0 on both Mac/Windows
If you want interoperability, you need to use versions that use the same
file format.
File format was the same from version 3.0 to 4.1, then it changed and the
price went up.
v. 3.0 runs great on Windows 2000 and Mac 9.2 and it only cost me $20 on
ebay.com as compared with the
latest 5.0 that is selling for around $250

IF Filemaker has problem importing:

 check to make sure your input and output files were DOS format
 try the Windows version; it seems more stable than the Mac

 My original InTouch data file had 6186 records
 For whatever reason, Filemaker Pro 3.0v4 for Macintosh could not import
my data file, which had 6591 records
    It only imported 534 records before claiming: "File is damaged...could
    not read disk"
    Another odd thing was that Record ID field showed that I had imported
    572 records; not 534
    I tried Filemaker for Windows 3.0 and it indicated 532 records in the
    file
 I tried importing in Windows version instead and succeeded

//////////////  CROSS PLATFORM FILENAME ISSUES

    Due to cross-platform character conversion problems, avoid the
    following characters in a filename:
         | \ " < > / ?
    Spaces are ok. I don't know about tabs, newlines but everything else
    should be ok.
    Avoid all high ascii characters, of course, such as:
        Mac option-8 bullet () becomes -> %MA5 in linux

    File saved from Mac to linux, via DAVE-mounted volume appears as
            -> %2A in linux/Windows-Samba
           | -> %7C
           \ -> %3A
           " -> %22
           < -> %3C
           > -> %3E
           / -> %2F
           ? -> %3F

    File created as in linux appears as
             hot -> HO~G2 in both Mac and Windows

//////////////  FILEMAKER SCRIPTING NOTES

 In scripts trying to match carriage return, use ascii 166 or \xA6 (hex) -
Option-7 in macintosh
    \r, \n are not recognized by FMPro; literal return inside a script is
    interpreted as a blank space

 Fields used in popup menu lists must be INDEXED.
    Calculated fields that reference related fields or globals cannot be
    indexed
    Therefore, instead of using a calculated, non-indexable field in a
    popup menu, use a simple field that is
    calculated and populated by a script - this requires invoking the
    script so values may get out of date.