To: ick-discuss@ick.liw.fi
From: Steve Kemp <steve@steve.org.uk>
Date: Sat, 09 Feb 2019 05:07:55 +0000
Message-ID: <1549688875.1308.1@ssh.steve.org.uk>
In-Reply-To: <5e2b278847b76dd0311d5050b3455b12b4dd3077.camel@liw.fi>
References: <5e2b278847b76dd0311d5050b3455b12b4dd3077.camel@liw.fi>
Subject: Re: Plan for using Muck for the Ick controller
Precedence: list
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi

> * The controller will reconstruct the whole build log when it's
>   requested for a complete log, by fetching all log objects that refer
>   to the specified build, and catenating the log output snippets in
>   order of the sequence number.

  Assume you're building a kernel, which might spit out 400,000 lines
 of build-log.  Assume each new update, due to buffering, or sizing,
 is 10 lines.

  To get the whole build-log you're going to need to:

    * Get the first piece.
    * Get the next child.
    * Get the next child.
    * Get the next child, 39,997 times more.

  That seems like it will scale terrible.  The only potential win I can
 see here is imagining that you might want to view the output via a
 brower and you'll probably only care about the LAST 100 lines, not the
 FIRST.  (i.e. The bit which will typically contain "built blah", or
 "build failed".)

  I'd be tempted to write the log locally, then upload as one massive
 object when complete.  Sure it might be big, but it's more efficient
 to stream a 1Mb HTTP transfer than to keep repeating HTTP-calls to
 get the next bit.

  (For a web-case you can use AJAX to stream on-scroll.  But for email
 output you'll want the whole thing to send..)


Steve
-- 
https://www.steve.org.uk/

_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi