Welcome Guest! Log in
×

Notice

The forum is in read only mode.
Due to some maintenance operations, stambia.org will be switched to read-only mode during the 13th November. It will be possible to read and download, but impossible to post on the forums or create new accounts. For any question please contact the support team.

Topic-icon Question Engine Exception : Impossible to replace file:...temp/_xcCTwy2YEeaUi6BCFjsWuw.xml

More
11 Jun 2016 23:24 #1 by mdahlman
mdahlman created the topic: Engine Exception : Impossible to replace file:...temp/_xcCTwy2YEeaUi6BCFjsWuw.xm
Here's the background:
I'm executing a mapping using the template "INTEGRATION Rdbms to Wsdl". The mapping is part of a process executed as a delivery where 10 sessions run in parallel.

Here are the errors:
Engine Exception : Impossible to replace file:/Applications/Stambia/StambiaDesigner18.1/stambiaRuntime/temp/_xcCTwy2YEeaUi6BCFjsWuw.xml
Engine Exception : /Applications/Stambia/StambiaDesigner18.1/stambiaRuntime/temp/_xcCTwy2YEeaUi6BCFjsWuw.xml (No such file or directory)

Troubleshooting:
I watched the files being created here: StambiaDesigner18.1/stambiaRuntime/temp
I see many files created with names like this: 1465663175019.temp.1._xcCTwy2YEeaUi6BCFjsWuw.xml
But I see only one file with a name like this: _xcCTwy2YEeaUi6BCFjsWuw.xml

It appears that most steps in the template are creating temp files based on the session ID. This is good.
But one step in the template is using a filename based on the delivery name. This is bad because the parallel sessions interfere with each other.
When I look at the variables for this step I can see the issue:
IN_FILE_NAME: /Applications/Stambia/StambiaDesigner18.1/stambiaRuntime/temp/_xcCTwy2YEeaUi6BCFjsWuw.xml
I can set the RECEIVED_MESSAGES_FOLDER and SENT_MESSAGES_FOLDER. But I don't see how to set the IN_FILE_NAME.

Probably I'm failing to set some parameter correctly for the integration template. But when I set "Out File Name" I get errors. Here I set the out file name to "abc":
Engine Exception : null/1465673579932.temp.1.abc (No such file or directory)

I see that my param was taken into account. But I don't know how to control that null at the beginning of the file path.

My solution:
I edited "INTEGRATION Rdbms to Wsdl". I changed the Code for the XSL variable "fileName"
original (whitespace added for readability in this post):
md:ifEmpty(
  md:paramValue($this,'OUT_FILE_NAME'),
  $REF/ref:target()/tech:xmlFilePath(),
  concat('${/CORE_TEMPORARY_FOLDER}$/',$REF/ref:target()/@id,'.xml')
)
modified (whitespace added for readability in this post):
md:ifEmpty(
  md:paramValue($this,'OUT_FILE_NAME'),
  $REF/ref:target()/tech:xmlFilePath(),
  concat(
    '${/CORE_TEMPORARY_FOLDER}$/',
    concat('${/CORE_SESSION_ID}$',$REF/ref:target()/@id,'.xml')
  )
)

Also, I changed the Triggering Behavior on the link to FileDelete. I made it non-mandatory from SqlToXml to FileDelete (mandatory didn't really make sense here). And I changed the outbound link from "Call WebServices" to go to FileDelete instead of going directly to "Unlock of CDC table". That way my files get cleaned up. When the same file name gets reused as it was in the original, then keeping it forever isn't really an issue. But with my change I would accumulate temp files.

Can you please validate this fix? Is this a good way to fix the template's inability to handle concurrent executions? Is there a better or easier solution?
More
12 Jun 2016 05:23 #2 by mdahlman
mdahlman replied the topic: Engine Exception : Impossible to replace file:...temp/_xcCTwy2YEeaUi6BCFjsWuw.xm
Update:
I found a surprising side-effect in a different job. In Mapping2 I have a dummy SQL source mapping to a web service using "INTEGRATION Rdbms to Wsdl". Then the mapping loads into a SQL table using "Load Wsdl to Rdbms".

I got errors indicating that the expected file wasn't found. I didn't realize this implicit link between "INTEGRATION Rdbms to Wsdl" and "Load Wsdl to Rdbms". By changing the temp file name in one, it causes the other to fail.

So I made the equivalent change in "Load Wsdl to Rdbms"
original:
md:ifEmpty(md:paramValue($this,'inFileName'),$REF/ref:source()/tech:xmlFilePath(),concat('${/CORE_TEMPORARY_FOLDER}$/',$REF/ref:source()/@id,'.xml'))
modified:
md:ifEmpty(md:paramValue($this,'inFileName'),$REF/ref:source()/tech:xmlFilePath(),concat('${/CORE_TEMPORARY_FOLDER}$/',concat('${/CORE_SESSION_ID}$',$REF/ref:source()/@id,'.xml')))

This seems to work.

But I'm stuck with the problem of accumulating lots of temp files. "INTEGRATION Rdbms to Wsdl" doesn't appear to have an option to clean up the temp files. Am I missing some option? And "Load Wsdl to Rdbms" has the option "Clean Temporary Objects", but this does not delete the temp files.

So my remaining questions are:
  • Am I likely find any other bad side effects from my tweaks?
  • Is there any standard way to purge my temp files?
More
13 Jun 2016 11:49 #3 by Thomas BLETON
Thomas BLETON replied the topic: Engine Exception : Impossible to replace file:...temp/_xcCTwy2YEeaUi6BCFjsWuw.xm
Hi Matt,

Normally if you set the same full file path in both "INTEGRATION Rdbms to Wsdl/OUT_FILE_NAME" and "Load Wsdl To Rdbms/IN_FILE_NAME", then the mapping will work fine without changing the template.
Example of full path: ${/CORE_TEMPORARY_FOLDER}$/abc.${/CORE_SESSION_ID}$.xml
However you are right : these temp files remain after execution.

Note: I suppose you cancelled the new link from "Call Webservices" to "File Delete". Indeed as you understood "Load Wsdl To Rdbms" will need this file.

In reply to your last two questions:
- I don't think so.
- For now you need to setup some custom process or additional template steps for this.

Your template modification is interesting, I will discuss it with my R&D colleagues.
I'll come back to you soon.
More
13 Jun 2016 22:18 #4 by mdahlman
mdahlman replied the topic: Engine Exception : Impossible to replace file:...temp/_xcCTwy2YEeaUi6BCFjsWuw.xm
"Note: I suppose you cancelled the new link..."
Yes. You are exactly right about that.

Ah... I think I understand better now. I wanted to use "abc" as my file name for a quick test. This crashes. But perhaps my test would have worked with this:
${/CORE_TEMPORARY_FOLDER}$/abc
or even a fully hard coded path might have worked for my quick test:
/Applications/Stambia/StambiaDesigner18.1/stambiaRuntime/temp/abc.xml

Perhaps "Out File Name" would be clearer as "Out File Full Path".

If I had understood that, then I would not have needed to change the template. Then I could have used ${/CORE_SESSION_ID}$ (or some other variable) to keep the parallel sessions from colliding.

Thanks for confirming those points. I'll be interested in future updates to the template. But for now it's working well. And we'll need to handle the file cleanup manually. That shouldn't be too hard. (But it would be nice to have the templates include this option.)
More
14 Jun 2016 16:36 #5 by Thomas BLETON
Thomas BLETON replied the topic: Engine Exception : Impossible to replace file:...temp/_xcCTwy2YEeaUi6BCFjsWuw.xm
Thanks for the feedback :)

You got it: ${/CORE_TEMPORARY_FOLDER}$/abc or any absolute path will do the job.

Property names cannot be changed. If we change the name, then existing mappings will lose their property values.
That being said, we have a work in progress: template documentation! When it's done, you will get tooltips on properties and general documentation generated from the template's description.

Concerning the template modifications: they will be done (for better parallelize capability & better cleaning), this is logged.
Thanks again!