FAQ: ContentConnector General

[CommunityJedi22]

This article provides you with general frequently asked questions for ContentConnector. Please see the links below and post any questions in the comments area.  If you cannot find what you are looking for, please Ask a Question HERE for DataSource.  

How to update from FTP to SFTP?

• If you are currently using ContentConnector in FTP mode (ftp.cs.1worldsync.com) and need to update to SFTP mode (sftp.cs.1worldsync.com), please follow the steps provided in this Mini Guide.

Can ContentConnector send an email if it has not succeeded?

• Yes. You can configure configuration\log4j.xml to have an SMTPAppender that only responds to FATAL log messages.

Can I run ContentConnector from a different directory?

• ContentConnector configuration uses relative paths by default. This makes it easier to deal with the variety of installation locations and systems that our customers use. The recommended method is to have your scheduler use ContentConnector root directory as the starting or working directory. The next easiest method is to run ContentConnector in a batch/script file. The scripts first task should be to change to ContentConnector root directoy. The last method is to alter the configuration files to use absolute paths instead of relative paths.

Can I run the ContentConnector on the MariaDB platform?

• The ContentConnector is compatible with the MariaDB. You would need to enable the MySQL settings in the contentconnector.xml file as follows:

  <Schema definition="configuration/DataSource/DataSource-MySQL.xml"/>

  <!-- MySQL -->

  <Databases Type="MySQL4.0CompatibilityMode">

  <Driver>com.mysql.jdbc.Driver</Driver>

  <ServerURL>jdbc:mysql://SERVER:3306/DATABASE</ServerURL>

  <ProductDataDatabase name="DATABASE" repositoryTablePrefix="" stagingTablePrefix="in_"/>

  <Login User="USERID" Password="PASSWORD"/>

  <TableType>MyISAM</TableType>

  <Loader>Local</Loader>

  </Databases>

Can I run the ContentConnector on the Microsoft Azure platform?

• The ContentConnector is compatible with the Microsoft Azure platform. You would need to make the following changes to the contentconnector.xml file.
  
  • <loader>BCP</loader> should be used instead of <loader>Native</loader>.
  • <option>ConnectorStatus</option> should be commented out.

Can I specify a download.zip to use?

• Normally, importnodownload (and legacy commands fullnodownload or incrementnodownload) trigger loading the files from the LocalUnzippedPath directory. However, you can tell ContentConnector to use any copy of download.zip. ContentConnector will unzip that file into the LocalUnzippedPath directory and process it normally. To do so, run ContentConnector as follows: java -jar dataSource-2.2.x.jar datasource configuration\ContentConnector.xml importnodownload path\to\my\otherdownload.zip

Can I use SFTP with the ContentConnector?

• Yes, you can use SFTP with the ContentConnector.

  The 1st step is to enable the following option in the contiguration.xml file.

  <!--Option>Secure FTP</Option>

  The 2nd step is to add the 1WS SFTP server to your known_host file.

  Unix : by connecting manually using the sftp command line tool: sftp sftp.cs.1worldsync.com

  • The default known hosts file location is ~/.ssh/known_hosts, if it does not exist you can either get it created by the above command or create it in the current folder by other means

  Windows: installing SSH from OpenSSH for Windows running in a new DOS window:

  • cd <connector install location>
  • ssh-keyscan -t rsa sftp.cs.1worldsync.com > known_hosts

How do I configure a SOCKS proxy that I have?

• The ContentConnector has been tested to successfully work with SOCKS proxy 4 and 5. In the ContentConnector.xml the following line should be set:
  <Proxy Host="MYPROXYHOST" Port="MYPORT" Username="" Password=""/>

  Replace MYPROXYHOST with the existing SOCKS proxy address and replace MYPORT with the SOCKS proxy port. Currently, the SOCKS username and password are unused and therefore, should remain blank. This line should also be placed after the DigitalContent sections, but before the Thumbnails section.

How do I configure ContentConnector to send an email when it exits with the “FATAL” error?

• Contact your local network administrator and request the following:

  • Mail account which ContentConnector can use to send emails.
    This information will be used for the ContentConnector mail address, ContentConnector login and ContentConnector password.
    
    
  • Mail server host name and port.
    This information will be used for SMTP Server Host and SMTP Server Port
    
    
  • Email address.
    It will be used as notification email address.

When the information is received, please follow the next steps to configure the notification emails:

• Open log4j.xml file (located in the directory "configuration") in a text editor.
• Uncomment the section starting with <appender name="MAIL">
• Update the parameter value with SMTP Server Host in <param name="SMTPHost" value="smtp.host.net" />
• Update the parameter value with SMTP Server Port in <param name="SMTPPort" value="25" />
• Update the value smtpusername with the ContentConnector login in <param name="SMTPUsername" value="smtpusername" />
• Update the value smtppassword with the ContentConnector password in <param name="SMTPPassword" value="smtppassword" />
• Update the parameter value with the ContentConnector mail address in <param name="From" value="connector@yourdomain.com" />
• Update the parameter value with the notification email address in <param name="To" value="customer@yourdomain.com" />

• Uncomment <appender-ref ref="MAIL"/> located in the “root” section.
• Save log4j.xml file.

  Then the notification email will be sent to the address specified in the parameter "To" when the ContentConnector has a “FATAL” error.

How do I configure ContentConnector.xml file

• When using with DataSource, ContentConnector requires two configuration files: ConnectConnector.xml and DataSource\DataSource-DATABASE.xml. The DataSource-DATABASE.xml file contains platform dependent SQL instructions, and is currently offered in four versions: DataSource-SQLServer.xml, DataSource-MySQL, DataSource-Oracle and DataSource-PostGreSql.xml. The configuration\ContentConnector.xml file contains the setup information for ContentConnector. Locate and edit the ContentConnector.xml file and make the edits to align the behavior of the application to your needs. More details can be found in the integration guide

How do I fix my thumbnails from appearing jagged or with defects?

• An option is to install the native JAI library. Install the JRE version of the JAI library for your platform.

How do I know what has changed from one version of ContentConnector to the next?

• You may see what has been added, in each version of ContentConnector, by reviewing the file "changes.txt" which is in the same folder of ContentConnector itself.

How do I prepare ContentConnector to use an absolute path?

• You need to edit the ContentConnector.xml file. The data file locations, the digital content then the upload file locations must all be changed to absolute paths. Next, the log4j.xml file must be edited. The locations of datasourceLog.txt and datasourceLog.html must be changed to absolute paths. Finally, when the ContentConnector is run, both the JAR file and configuration file must be specified in terms of absolute paths.

How do I run the ContentConnector in debug mode?

• To run the ContentConnector in debug mode, make the following update:

  • In the configuration\log4j.xml file, replace <priority value="info"/> with <priority value="debug"/>.

  With this option enabled, the ContentConnector will log additional debug information and load it into the logs\log.txt file.

How often is ContentConnector upgraded?

• Each time 1WorldSync adds a new feature (alternative categorization, lifecycle...), the connector may also require an update to integrate the new tables. It may also be updated due to various changes within the different operating systems.

How often should I run ContentConnector and at what time?

• We are not able to guarantee/determine an exact delivery time for the files. This will vary based on that cycle's load. Only one download.zip can be delivered during one distribution cycle and a cycle may normally last one to two days. Therefore, running it once a day should be fine. You may also choose to run the processing of the download.zip several times and at different hours, in a day, if you want it to be processed on the same day it is received.

How to upgrade to the latest ContentConnector version?

• Here are the upgrade steps:

  For version 2.2.3.4 and higher

  • Download the latest ContentConnector version here.
  • Copy the JAR file, the schema file (configuration\DataSource) and the config file (ContentConnector.xml) from the new version and replace these files in the old version.
  • Modify the config file (ContentConnector.xml) according to your account settings and enable the new options if needed.
  • Run the Connector as usual

  For versions before 2.2.3.4

  • Download the latest ContentConnector version here.
  • Copy the JAR file, the schema file (configuration\DataSource) and the config file (ContentConnector.xml) from the new version and replace these files in the old version.
  • Uncomment CompatibleSchema option in the config file (ContentConnector.xml)
  • Modify the config file (ContentConnector.xml) according to your account settings and enable the new options if needed.
  • Run ContentConnector with create command
  • Run the ContentConnector as usual

I requested a full file. Do I need to make any adjustments to ContentConnector?

• If you are running ContentConnector 2.2.3.4 or later then you do not need any adjustments. ContentConnector detects the file type automatically and processes it accordingly.

  If your version is 2.2.3.3 or below, make sure that the option “CheckIsFullDownload” is enabled in the config file (ContentConnector.xml). ContentConnector will then detect the file type automatically and process it as a full file.
  If you are on a version 1.6 or below please contact your Customer Support Team for upgrade.

I set my HTTP proxy settings on the command line, why is it still not working?

• On some systems, the Java applications don't seem to pick up the proxy settings from the command line. Instead, find the "net.properties" file on your system and edit the settings from there.

Why am I having to constantly reset table permissions?

• This may be a problem if you assign users table level permissions on Oracle or SQL Server. For performance reasons, the ContentConnector sometimes does table swapping. Instead of updating tables, it will build a new table and replace the old table with the new table. This means that the user permissions are not set for the new table. The solution is to set all permissions at the database level. Assign users to a role, and then allow the role to determine what permissions the users have (select only, owner, write access, etc.). This way, the same permissions will apply to any new tables.

Why does the log state that "An increment has been missed"?

• The ContentConnector (starting from the version 1.6) detects missed increments. The config file (ContentCoonnector.xml) has the option "CheckMissedIncrement" automatically enabled. It does this by comparing staging cds_Catalog and both staging and production cds_Prod. All products found in staging cds_Catalog must be found in either the current production cds_Prod or the staging cds_Prod.

  If you get this error, you would either need to manually download the missed download.zip file and all other download files delivered after that from the Archive directory and run the Connector in nodownload mode or you may request the full file delivery via the Extranet or by sending the request to Customer Support team.

Why don't accents and other special characters show up correctly?

• This is a result of the default character set running on most Linux environments. Usually, the default character set is UTF-8. However, most of the DataSource data files are ASCII, not Unicode. Therefore it should work in Windows, which has a default character set of Windows-1252. On Linux, the most similar character set is ISO-8859-1. Java provides an easy way to specify an alternate character set. Run ContentConnector with the following command:
  java -Dfile.encoding=ISO-8859-1 -jar ...

What to do when Java is out of memory

• If you have an error message similar to this “java.lang.OutOfMemoryError: Java heap space“, you can increase the heap size allocation for a Java application by adding extra parameters in the command line

  • For 32 bit systems
    java -Xms512m -Xmx2g -jar contentConnector-2.x.x.jar datasource configuration/ContentConnector.xml import

  • For 64 bit systems
    java -Xms512m -Xmx4g -jar contentConnector-2.x.x.jar datasource configuration/ContentConnector.xml import

• -Xms512m: sets the initial (minimum) heap size to 512 megabytes (MB)

• -Xmx2g: sets the maximum heap size to 2 gigabytes (GB)