Welcome Guest! Log in
×

Notice

The forum is in read only mode.
Due to some maintenance operations, stambia.org will be switched to read-only mode during the 13th November. It will be possible to read and download, but impossible to post on the forums or create new accounts. For any question please contact the support team.

Topic-icon Question Limit number of rows in a mapping

More
09 Jun 2016 00:09 #1 by Shahin
Shahin created the topic: Limit number of rows in a mapping
Bonjour,
I have a mapping from a DB to a JSON webservice. I would like to only load 100 rows at a time, but I cannot figure out how to do this? Is there a filter that can do this?

Thanks!
More
09 Jun 2016 16:39 #2 by Cyril Dussud
Cyril Dussud replied the topic: Limit number of rows in a mapping
Hi,

In a Mapping a target WebService is invoked for each value of the repetition key.
So if you have 1000 source rows, the target WebService will be called 1000 time sequentially.

Can you give a little more detail on what you want to accomplish ?
If you want the Mapping to load only 100 rows, you could filter the source
More
09 Jun 2016 17:47 #3 by Shahin
Shahin replied the topic: Limit number of rows in a mapping
We have product data in Semarchy and need to export that to Netsuite via JSON WS. Our ultimate goal is to do this as fast as possible. In the past, simply running the mapping as is with not filters on the entire data set took days to complete. I was hoping to run the mapping multiple times in parallel to increase the throughput. Is there a better way to increase the performance?

How would I create a filter on the source? I only see a way to filter at the attribute level. Please be specific in your answer and be aware because of the fact I am in the US there are time zone differences.

Thanks for the help.
More
11 Jun 2016 18:55 #4 by mdahlman
mdahlman replied the topic: Limit number of rows in a mapping
Let's close this thread. I'm working with Shahin.
I know how to query to get N records and then submit them via web services calls in M batches. Basically it's just a SQL Operation bound to the execution of a Delivery.

But I'll open a new thread with a more precise question about an issue I'm seeing with running these sessions in parallel.
More
13 Jun 2016 09:26 #5 by Thomas BLETON
Thomas BLETON replied the topic: Limit number of rows in a mapping
Thanks Matt, indeed making batches of rows is usually done with SQL.
For your information, we plan to improve our WS features so that one can specify parallelization params.