(v13) HqnImpose2 Data sources

This page applies to Harlequin v13.1r0 and later; both Harlequin Core and Harlequin MultiRIP.

As from Harlequin 12 the value of a number of mark parameters can be set in more flexible ways than in previous versions. This is achieved by providing a data source dictionary value, which is resolved to generate the required result (for example):

            /Text <<
                        /Type /Dynamic
                        /Source /Names
                        /Key /Char1
                      >>

The dictionary must contain a Type entry, and additional entries depending on the value of Type :

Table: Data source Type entries

Dynamic source types

If a data source with Type=Dynamic is used, the Source key is used to select an entry from the DynamicSources dictionary in the current mark. Each entry in the DynamicSources dictionary is used to specify a source for data records; a data source references a dynamic source, and, where appropriate, selects a field from that data record.

Each dictionary value within DynamicSources must contain the SourceType key with a name value, and the name must either be one of the following, or a value added as described in Adding new data source types.

Type	Description and other keys
/CSV	This dynamic source reads records from CSV files. A new record is read from the file for every Geometry that uses this source. CSV files: —Must conform to RFC 4180 with respect to quoting of special characters; this requires that fields containing double quotes or delimiters must be surrounded by double quotes, and that any double quote within a field must be escaped by preceding it with another double quote. —May use any encoding for which the double-quote is represented as 34 (0x22), and where no byte in a multi-byte character uses that value or the same value as the character code for the selected delimiter. This means ASCII, ISOLatin1, UTF-8, and so on are all acceptable when used with common delimiters such as comma, tab, semi-colon, or pipe. The font encoding to be used to show any text provided through the CSV must match that provided in the CSV file. Note that ControlBarcode marks require input data to be provided in UTF-8 encoding (and that 7-bit ASCII is a pure subset of UTF-8). —Must not include end-of-line within any field; such a field would not have meaning in any use case in HqnImpose2 anyway. —Must include a header line that defines the name for each field within the records —Must not include the Excel-specific sep= metadata at the start of the file. —From Harlequin v13: May contain a UTF-8 BOM at the start of the file. Use of BOMs for other encodings, or of a UTF-8 BOM for Harlequin versions before 13.0r0, prevents the header for the first column of data from being read correctly. Entries within the dictionary within DynamicSources other than SourceType are: /SourceFile —Required if Context is not provided, ignored if Context is specified; string. The file name and path at which the CSV file to be read can be found using PostScript file name formatting. /Context —Optional; CSV reading context. If not specified it will be created automatically by opening the file specified by SourceFile . /Delimiter —Optional, integer between 1 and 255. The character code value for the delimiter used to separate fields within each record. Defaults to 44 (0x2C) for a comma (,). A CSV dynamic source reads records from the CSV file. A data source whose Source entry refers to a CSV source used in a mark must select an individual field from that record using the Key entry: /Key —Required, name or data source returning a name; the name of the field that should be read from the CSV record to provide data for this mark. This must exactly match a field within the header line of the CSV file, which is case sensitive. If a CSV data source is exhausted (end of file is reached), it automatically preventa any further overlay marks being drawn in this and any subsequent Geometries and aborts the job at the end of the current sheet. The CSV reading uses the HqnCSV procset. This procset is only supported for use with overlays using HqnImpose2 as described above. Other uses are not supported.
/Counter	This dynamic source maintains a counter that can be incremented after each geometry that uses it. Additional entries within the dictionary within DynamicSources when SourceType =/Counter are: /Value —Required, integer. The starting value for this counter (that is, the value to be used in the first mark that references it). /Increment —Optional, integer. The amount by which the counter value should be incremented after every Geometry that uses it. Default=1. Negative values may be used to decrement the value. /Every —Optional, positive integer. Use a number greater than 1 to increment the counter after some number of Geometries that use it, instead of every Geometry. This may be used for batch numbering, for instance. Default = 1 . /ResetOver —Optional, Integer. Specify ResetOver if the counter should be reset to a specific value after it reaches or exceeds ResetOver. /ResetTo —Optional, integer. If the counter is reset from ResetOver or ResetUnder it sets to ResetTo . Default = 1 . /ResetUnder —Optional, integer. Equivalent to ResetOver , for use when Increment is negative.

Table: SourceType key names

Example use of /DynamicSources :

  <<
  ...
  /MarkSets [
      <<
      /DynamicSources<<
        /Names<<
          /SourceType /CSV
          /SourceFile (names.csv)
        >>
        ...
      >>
      ...
      /SetMarks [
          <<
          /Enabled true
          /MarkType / ControlGraphicText
          /Text <<
            /Type /Dynamic
            /Source /Names
            /Key /SampleText
          >>
          /X 28.8
          /Y 792
          /MarkWidth 252
          /TextFont /Helvetica-Bold
          /TextSize 80
          /TextAlign /Center
          /TextFitting /Scale
          /TextCS [ /DeviceGray ]
          /TextColor [ 0 ]
          /TextRendering /Show
        >>
        ...
      ]
      ...
    >>
    ...
  ]
  ...
  >> /HqnImpose2 /ProcSet findresource /HqnInitOverlay get exec

String or string sources

A number of String parameters may be specified using a data source dictionary instead of specifying a string directly. This includes:

• Text in a ControlText mark (see ControlText in (v13) Drawing standard HqnImpose2 marks).
• Text in a ControlGraphicText mark (see ControlGraphicText in (v13) Drawing standard HqnImpose2 marks).
• Data in a ControlBarcode mark (see ControlBarcode in (v13) Drawing standard HqnImpose2 marks).
• FileName in a ControlFile mark (see ControlFile in (v13) Drawing standard HqnImpose2 marks).
• Key in a Type=Dict data source (see Type=Dict in Other data sources (below). The data source must return a string.

A string data source may also be provided as an array of strings and data source dictionaries. The final string is formed of the values of all of them concatenated.

A data source dictionary used to specify a string may also contain the following additional keys:

/Suffix —Optional, string. Only acted on when the data source dictionary is included in an array of sources. The string supplied is added after the primary return value of the data source if that return value is not an empty string.

/PadLength —Optional, integer. If the string returned by the data source is less than PadLength characters then additional characters are added at the front to pad it to the required length. PadLength should only be used for single-byte text encodings. Strings that are as long or longer than PadLength are not affected by padding.

/PadChar —Optional, string or integer. The character specified by the integer or the first character of the string is repeated as required to pad the string to the desired length. Default=48 (‘0’).

Example:

              /Text [ (TO: )
                <<
                /Type /Dynamic
                /Source /Names
                /Key /First
                /Suffix ( )
              >>
                <<
                /Type /Dynamic
                /Source /Names
                /Key /Middle
                /Suffix ( )
              >>
                <<
                /Type /Dynamic
                /Source /Names
                /Key /Last
              >>

Other data sources

The TextFont key in a ControlGraphicText mark may be provided as a data source that returns a name. This is very similar to a string data source, except that the array form is not allowed.

A data source that returns a dictionary may be used as the value of the Dict key in a Type=Dict data source.

The dict returned is not treated as a data source.

A data source that returns an integer may be used as the value of the Index key in a Type=Array data source.

Nesting data sources

In some cases it can be useful to nest data sources, but this should be done with care. For complex cases, it is often better to calculate the data required upstream and to deliver it pre-composed in the CSV data.

Examples:

Choose a random entry in an array for the color of a ControlGraphicText mark:

            /TextColor <<
              /Type /Array
              /Array [[ 1 0 0 0 ] [ 0 1 0 0 ] [ 0 1 1 0 ]]
              /Index <<
                /Type /Random
              >>
            >>

Translate from CSV data that records days of the week as “Mon”, “Tue” , and so on into French:

            /MarkContentDicts <<
              /FrWeekDays <<
              /Mon (Lun)
              /Tue (Mar)
              /Wed (Mer)
              …
              >>
            >>
            …
            /Text <<
              /Type /Dict
              /Key <<
                /Type /Dynamic
                /Source /Names
                /Key /Day
              >>
              /Dict /FrWeekDays
            >>

Adding new data source types

You may adda new data source type if required (for example, to read data directly from a database, or from some other data format such as JSON). A data source type encapsulates both the format of the data itself and the channel through which it is read. If you don’t plan to read from a disk file you may well wish to start by adding a device and/or filter to the RIP to provide a channel to access the data through. For more information see (v13) Files, devices, device types, channels, and channel classes and (v13) Filters).

To add a new data source type you must add procedures to the DynamicSourcesInit , DynamicSourcesUpdate , and LoadDynamicSource dictionaries in the HqnImpose2 procset. This is usually best done by saving them into a file named SW/Usr/CustomImpose2 to avoid the need to edit the procset directly.

The keys for the new procedure entries in those dictionaries must be the name that is used for SourceType in the configuration.

The procedure in DynamicSourcesInit is called during initialization of HqnImpose . It must do all initialization required for the data source, including opening files and creating structures as required. It must then read the first data record, which may be done most easily by calling the relevant procedure from DynamicSourcesUpdate.

When called, the relevant entry from the DynamicSources dictionary is on the top of the dictionary stack. All data required to initialize and process the dynamic source that may vary by job should be delivered in that dictionary; you can add whatever entries you require into it within reason. All values stored by the procedure should simply be defined into this dictionary, to ensure that using the same data source type from multiple DynamicSources dictionaries act independently.

The procedure must leave the operand and dictionary stacks, and the save level unchanged. All data structures created must be in global VM.

The procedure in DynamicSourcesUpdate is called for each MarkSet, to load the next record from the data source.

Just as for DynamicSourcesInit, when called, the relevant entry from the DynamicSources dictionary is on the top of the dictionary stack; the procedure must leave the operand and dictionary stacks, and the save level must remain unchanged. The data structure created depends on the kind of data to be returned; it is normally a dictionary containing all fields in the record, and it must be in global VM.

The procedure in LoadDynamicSource is called for each Mark that uses that data source within each MarkSe . It should return a single field from the record loaded in the last call to the DynamicSourcesUpdate procedure. We recommend that you mirror the requirement for a /Key entry in the DynamicSources dictionary from the CSV data source type unless you have good reasons to use a different approach, and use that Key value to identify the field.

As before with the other two procedures, when called, the relevant entry from the DynamicSources dictionary is on the top of the dictionary stack; the procedure must leave the dictionary stack, and the save level must remain unchanged. The only change to the operand stack must be the addition of the field value to be returned.

For more information see the CSV entries in the DynamicSourcesInit, DynamicSourcesUpdate, and LoadDynamicSource dictionaries in the HqnImpose2 procset for an example implementation.

The requirement for data to be stored in global VM implies a need to re-use as many strings, arrays, and dictionaries as possible to avoid a build up of global VM usage.