How to see if what a specific load process is doing?

Question

Hi all, having a little bit of a problem in that I have a load (not replication) job that seems to be hanging (not failing) but also not loading any data into HANA or not finishing.
- Attempted to load table STXL in full, 440k or so rows loaded into HANA
- Created an include for BOR event to filter all but 1 record, worked fine in about 3 minutes and 1 record present in HANA
- Added additional code to decompress a compressed column format, code shown below. This was tested in a 'standalone' fashion as a program in an ECC system, so this is migrated to SLT as an include. In this case, the load job is just "hanging", it's not failing or not finishing. When I look in the monitor, can see it running for the last say 15 minutes "In Process".
From here, no data ever makes it into HANA and the job in SLT doesn't seem to finish.
How to diagnose where this is getting hung up or where the problem is? Normally if the code is bad, I would get a hard error in the application logs, which I don't get here.
Code for include
*&  Include           Z_TEST_IMPORT_TEXT
types: begin of ty_stxl_raw,
         clustr type stxl-clustr,
         clustd type stxl-clustd,
       end of ty_stxl_raw.
DATA: lt_stxl_raw type standard table of ty_stxl_raw,
      wa_stxl_raw type ty_stxl_raw,
      lt_tline type standard table of tline,
      wa_tline type tline.
IF <WA_S_STXL>-TDNAME <> '0020874233'.
  SKIP_RECORD.
ENDIF.
clear  lt_tline[].
wa_stxl_raw-clustr = <WA_S_STXL>-clustr.
wa_stxl_raw-clustd = <WA_S_STXL>-clustd.
append wa_stxl_raw to lt_stxl_raw.
import tline = lt_tline from internal table lt_stxl_raw.
READ TABLE lt_tline into wa_tline INDEX 1.
<WA_R_STXL>-TEXT = wa_tline-tdline.

Answer 1

Frank, awesome to hear you worked through the issues.
Regarding DMC_STARTER, can you share how you were able to find the issue, specifically with such a large dataset, did you just step through until you hit a dump? When I referred to the documentation on debugging (that I had not yet created!), here is what I gathered just last week as a rough start. I'll probably repost this as a blog if you can confirm it is similar to your steps.
- Ensure however you would like to test has already been started
- For example if you want to test initial load, start the initial load and let it fail, and follow the steps below. When DMC_STARTER kicks up, it will start initial load again in debug.
- If you want to test replication phase (picking up logging table records to process), get a table to replication status, stop the master job, then make a change in the source to log new entries in the logging table and follow the steps below. When DMC_STARTER kicks up, it will start in replication phase, picking up logging table records for processing.
After the above is "set up" for testing, do the following, some of the functions will have different names based on your implementation.
- SE38, DMC_STARTER
- Use Migration Object Z_<TABLE_NAME>_<CONFIGURATION_ID>, Z_STXL_001 for example, access plan = 1, test mode = X
- /h to enable debug
- Find OLC function module call CALL FUNCTION olc_rto_ident, set breakpoint, continue to enter the FM
- Enter 1CADMC/OLC_100000000000*, find CALL FUNCTION '/1CADMC/IL_100000000000769', set breakpoint, continue to enter the FM
- Enter /1CADMC/IL_100000000000769, find PERFORM _BEGIN_OF_RECORD_0001_, set breakpoint, continue to enter
- Enter PERFORM _RULE_BOR_C., the include code should be found here and you can step through.
Regarding your issues discovered
1) When I referred the code changes I need to tag on the other blog, I was referring to the following
- Check for <WA_S_STXL>-SRTF2 = '0' to only grab the first line (as you mention)
- The CLUSTR size as you mention is a helpful check.
- Also need to put a check above the processing of the record to be able to handle DELETE's, else you'll get a short dump. When a deletion occurs, only the key columns are passed through this logic, and when we are working (importing) on a non-key field it will of course be empty and cause a dump.
if <WA_S_STXL>-CLUSTD is not initial.
*Put Source fields into internal table for IMPORT statement to work on, main logic here
endif.
2) The specific texts we were targeting would never be this large, they only consist of one single line so we never hit this issue. In the case you exceed 345 lines, are you just truncating the remainder?
All in all, great stuff - good to see someone taking this concept further. If you want a deeper look into the folks who are doing much heavier text processing with additional logic (and much stronger ABAP'ers than I ), see Mass reading standard texts (STXH, STXL)
Happy HANA!
Justin