ESP scenario question!?

Question

Hi all,
I have the following scenario in mind, can someone tell me if ESP can support this scenario and which elements do I need to use in order to accomplish my scenario…
I want to use an input-stream to receive a lot of twitter messages which I want to analyse based on tag and type of message e.g. #complain, #angry, #sad and even #happy, #good etc.
I want to group all the message based on this type and after I have received for example 50 messages, I want to trigger a BPM process from ESP by using the Webservice Output Adapter.
I know how to use the input-stream and the Webservice Output Adapter. But how can I support the steps in between? I’ve used a Splitter to search the twittertext on keywords and define the path for further actions.
The first step after the splitter is that the message will be received by a Window. The next thing that I want achieve is to store the messages within the window up till a count of 50. After that, all the messages within the window must be send to a Webservice Output Adapter for further actions (BPM process will be started).
In the next scenario, I have used a CSV Output Adapter instead of the Webservice. What happens when I test my ESP process? All the message will be send one by one to the CSV file instead of bundled by 50 .
Any help is welcome
Header 1
declare
  parameter integer threshold := 5;
end;
CREATE INPUT STREAM TwitterInputStream SCHEMA (
  tweetDate string,
  tweetTime string,
  tweetId string,
  tweetUser string,
  tweetText string );
CREATE OUTPUT SPLITTER TwitterInputSplitter
AS WHEN lower ( TwitterInputStream.tweetText ) LIKE '%magnetron%' AND ( lower ( TwitterInputStream.tweetText ) LIKE '%klacht%' OR lower (
TwitterInputStream.tweetText ) LIKE '%failure%' OR lower ( TwitterInputStream.tweetText ) LIKE '%boos%' ) THEN MagnetronBadHashTag
WHEN lower ( TwitterInputStream.tweetText ) LIKE '%magnetron%' AND ( lower ( TwitterInputStream.tweetText ) LIKE '%happy%'
OR lower ( TwitterInputStream.tweetText ) LIKE '%good%'
OR lower ( TwitterInputStream.tweetText ) LIKE '%tevreden%')
THEN MagnetronGoodHashTag
SELECT * FROM TwitterInputStream ;
CREATE OUTPUT WINDOW Window1 PRIMARY KEY DEDUCED KEEP EVERY threshold ROWS AS SELECT 'Klacht' tweetType , * FROM MagnetronBadHashTag GROUP BY
MagnetronBadHashTag.tweetId ;
ATTACH OUTPUT ADAPTER File_Hadoop_CSV_Output1 TYPE toolkit_file_csv_output TO Window1 PROPERTIES dir = 'C:/Users/Gerritsen/Desktop' ,
  file = 'ESP_OUTPUT.csv' ;

Answer 1

Hi,
I guess you want the output in a bundle of 50 rows and not individual rows...
If that is so then the way to do is, it needs a bit of tweaking
1. Create a join as join produces rows in bundles. You can create a join by joining with some kind of trigger after every 50 rows.  If there is a window with 50 rows try to join it with a trigger stream and it should produce 50 rows in a bundle.
2. Create a splash stream and make sure you read 50 rows from the input window and send it out in one read in a continous output statement. SPLASH combines continous output and makes them as one bundle.
http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc01612.0514/doc/html/san1311804098405.html
Using itereators read window data and keep sending it to the output using output statement.
The mulitple outputs get combined to one bundle of data.
Thanks,
Geetha.