advice for speeding up parts of my script

Anything beyond the basics in using the LiveCode language. Share your handlers, functions and magic here.

Moderators: FourthWorld, heatherlaine, Klaus, kevinmiller, robinmiller

Post Reply
niconiko
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 32
Joined: Mon Jul 17, 2006 2:28 am

advice for speeding up parts of my script

Post by niconiko » Thu Apr 18, 2019 4:10 am

Astonished by how many seconds parts of my scripts are taking!

*************************************************************************
PART #1... which takes 10 secs, even after trying (a) "offset" instead of "is not among", and (b) replacing t_chars_allowed_for_Roman_reference with the actual set of chars:

Code: Select all

function is_it_Roman theIndexPageNumber 
   -- theIndexPageNumber  is either a four digit number or a roman number
   initiateTimer(482,"start")
   put g_pg_nr_spread_separator & ",.();-" into t_chars_possibly_contiguous_to_Roman_reference
   put g_Roman_digits & t_chars_possibly_contiguous_to_Roman_reference into t_chars_allowed_for_Roman_reference
   put "yes" into t_is_nr_Roman
   repeat for each char tChar in theIndexPageNumber 
      if tChar is not among the chars of t_chars_allowed_for_Roman_reference then
         put "no" into t_is_nr_Roman
         exit repeat
      end if
   end repeat
   initiateTimer(482,"close")
   return t_is_nr_Roman
end is_it_Roman
************************************************************

PART #2... which takes 10 seconds... no alternate ideas for "itemOffset":

Code: Select all

 initiateTimer(469,"start")
   if itemOffset(thePgNr,t_TCMS_italic_pg_nrs) = 0 then
   -- thePgNr = 1 to 3 digit number (Roman or Arabic), t_TCMS_italic_pg_nrs = comma-delimited list of 71 items (same format as thePgNr)
      put "non-italic," & t_pg_nr_with_zeroes & comma & t_pg_nr_with_zeroes_prevPage into t_result
   else
      put "italic," & t_pg_nr_with_zeroes & comma & t_pg_nr_with_zeroes_prevPage into t_result
   end if
   initiateTimer(469,"close")
*************************************************

PART #3... each initiateTimer id (364,372,375) = 8 seconds (so total of 24 seconds):

Code: Select all

-- STEP #2    Get the last parapraph of the previous page.
   initiateTimer(364,"start")
   put lineoffset(t_htmltTemplate_paragraphID_for_thePgNrPrevious_1st_paragraph,g_html_text) into t_htmltTemplate_paragraphID_for_thePgNrPrevious_1st_paragraph_lineNr
  -- t_htmltTemplate_paragraphID_for_thePgNrPrevious_1st_paragraph = a 4-6 char string
  -- g_html_text = 5015 lines / 1,300,393 chars of HTML (it's the text of an e-book)
   initiateTimer(364,"close")
   if t_htmltTemplate_paragraphID_for_thePgNrPrevious_1st_paragraph_lineNr = 0 then
      -- no paragraphs in previous page, ie, a blank page
      put "{EMPTY}" & tab & "{EMPTY}" & comma before t_result
   else
      put empty into t_LO_times_prevNr
      initiateTimer(372,"start")
      repeat with t_line_nr = (t_lineoffset_1st_value - 1) down to t_htmltTemplate_paragraphID_for_thePgNrPrevious_1st_paragraph_lineNr
    -- repeats about 31 times or exits
         put line t_line_nr of g_html_text into t_html_text_line
        -- t_html_text_line = a printed-book paragraph
         initiateTimer(375,"start")
         put offset(t_htmltTemplate_paragraphID_for_thePgNrPrevious,t_html_text_line) into tCharOffset
         initiateTimer(375,"close")
         if tCharOffset <> 0 then exit repeat
      end repeat
      initiateTimer(372,"close")
*************************************************

PART #4... the "repeat" part of a function called 3-8 times, some calls lasting 0 seconds, some totalling 3-13 seconds:

Code: Select all

   repeat with t_paragraph_nr = 1 to g_maximum_possible_number_of_paragraphs
     -- g_maximum_possible_number_of_paragraphs = 31 HTML <p> delimited "paragraphs"
      put t_htmltTemplate_paragraphID_for_thePgNr into t_htmltTemplate_paragraphID_temp
      replace "[PHYSICAL_PARAGRAPH_ID]" with t_paragraph_nr in t_htmltTemplate_paragraphID_temp
     -- t_htmltTemplate_paragraphID_temp = a 4-6 char string
      initiateTimer(338,"start")
      put lineoffset(t_htmltTemplate_paragraphID_temp,g_html_text,t_lines_to_skip) into tLO
   --   g_html_text = 5015 lines / 1,300,393 chars of HTML (it's the text of an e-book)
      initiateTimer(338,"close")
*************************************************

PART #5... some cases 0 seconds, others 8-9 seconds:

Code: Select all

   set the caseSensitive to true
            set the wholeMatches to true
            initiateTimer(180,"start")
            if offset(t_fixedPhrase,t_retrieved_HTML_paragraph_text) <> 0 then
                -- t_fixedPhrase = a multi-word string (about 1-5 words)
                -- t_retrieved_HTML_paragraph_text = an HTML paragraph of about the same char-length of a printed-book paragraph
               add g_weighted_value_fixedPhrase to t_weighted_value_TOTAL
            end if
            initiateTimer(180,"close")
FINALLY... my initiateTimer handler:

-- types = reset,start,close,summarize
-- id is the line number within the script

Code: Select all

on initiateTimer theIDandType
   put the itemDelimiter into tSavedDelim
   set the itemDelimiter to comma
   put item 1 of theIDandType into t_script_lineNr
   delete item 1 of theIDandType
   put theIDandType into t_type
   if t_type = "reset" then
      set the itemDelimiter to tSavedDelim
      put empty into t_timer_data
      put "Initiating timer..." into field "timer_summary"
      exit initiateTimer
   end if
   if t_type <> "summarize" then -- handle "start" and "close"
      put the wholeMatches into tSavedWholeMatches
      -- set the wholeMatches to true
      put 0 into tLinesToSkip
      repeat
         put lineoffset(t_script_lineNr,t_timer_data,tLinesToSkip) into tLO
         if tLO = 0 then 
            add tLO to tLinesToSkip
            exit repeat
         end if
         add tLO to tLinesToSkip
      end repeat
      put tLinesToSkip into t_line_nr_of_recentmost_scripLineNrID
      put the number of lines in t_timer_data into t_timer_data_lineCount
      put the last line of t_timer_data into t_timer_data_lastLine
      switch t_type
         case "start"
            switch t_line_nr_of_recentmost_scripLineNrID
               case 0 -- the very first record for this script-lineNr id, so put the data at the end
                  put t_script_lineNr & comma & the seconds & comma & "-," & cr after t_timer_data
                  break
               default -- put new line id'd already in the data immediately after that previous id line
                  switch
                     case t_timer_data_lineCount = t_line_nr_of_recentmost_scripLineNrID
                        put t_script_lineNr & comma & the seconds & comma & "-," & cr \
                              after line (t_line_nr_of_recentmost_scripLineNrID + 1) of t_timer_data
                           break
                        case t_timer_data_lineCount > t_line_nr_of_recentmost_scripLineNrID
                           put t_script_lineNr & comma & the seconds & comma & "-," \
                                 after line t_line_nr_of_recentmost_scripLineNrID of t_timer_data
                              break
                           case t_timer_data_lineCount < t_line_nr_of_recentmost_scripLineNrID
                              breakpoint
                              break
                        end switch
                  end switch
                  break
               case "close"
                  put the seconds into item 3 of line t_line_nr_of_recentmost_scripLineNrID of t_timer_data
                  break
            end switch
            set the wholeMatches to tSavedWholeMatches
         else
            -- Summarize
            delete the last char of t_timer_data
            put empty into t_timer_data_summmary
            -- breakpoint
            repeat for each line t_timer_data_line in t_timer_data
               put item 1 of t_timer_data_line into t_lineNrID
               put item 2 of t_timer_data_line into t_seconds_start
               put item 3 of t_timer_data_line into t_seconds_end
               if t_lineNrID is empty then
                  breakpoint
               end if
               put t_lineNrID & comma & (t_seconds_end - t_seconds_start) & cr after t_timer_data_summmary
            end repeat
            delete the last char of t_timer_data_summmary
            sort lines of t_timer_data_summmary international ascending by item 1 of each
            put "** TIMER DATA **" & cr & t_timer_data_summmary into field "timer_summary"
         end if
         set the itemDelimiter to tSavedDelim
      end  initiateTimer
Help please! Thanks

dunbarx
VIP Livecode Opensource Backer
VIP Livecode Opensource Backer
Posts: 10305
Joined: Wed May 06, 2009 2:28 pm

Re: advice for speeding up parts of my script

Post by dunbarx » Thu Apr 18, 2019 4:42 am

Hi.

A lot of code. A quick glance at it shows no obvious time consumers, like reading and writing into a field. But I may have missed something. I don't suppose the variable "theIndexPageNumber" is many megabytes long?

Have you tried putting breakpoints every couple of lines, or after every reasonable "section" of your handlers? When you execute, you can see which breakpoints appear instantly, and which appear after a while. Then you can drill down more finely within that slow portion of the handler until you find the line or lines that are the culprit.

What happens if you comment out intitiateTimer? This is sort of the same thing as above, to isolate sections of the handler to see what takes a lot of time.

Craig Newman

Post Reply