Date: Sun, 24 Feb 2019 19:59:58 +0200
From: Lars Wirzenius <liw@liw.fi>
To: ick-discuss@ick.liw.fi
Message-ID: <20190224175958.GC17934@exolobe1.liw.fi>
References: <5e2b278847b76dd0311d5050b3455b12b4dd3077.camel@liw.fi>
 <1549688875.1308.1@ssh.steve.org.uk>
 <20190216144009.GA5870@exolobe1.liw.fi>
 <1550678063.32630.0@ssh.steve.org.uk>
MIME-Version: 1.0
In-Reply-To: <1550678063.32630.0@ssh.steve.org.uk>
User-Agent: Mutt/1.10.1 (2018-07-13)
Subject: Re: Plan for using Muck for the Ick controller
Precedence: list
Content-Type: multipart/mixed; boundary="===============0442428648705839811=="
Mime-version: 1.0
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi


--===============0442428648705839811==
Content-Type: multipart/signed; micalg=pgp-sha512;
 protocol="application/pgp-signature"; boundary="orO6xySwJI16pVnm"
Content-Disposition: inline


--orO6xySwJI16pVnm
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Feb 20, 2019 at 03:54:23PM +0000, Steve Kemp wrote:
> > (Sorry about taking so long to answer: I was travelling for work.)
>=20
>   No worries.
>=20
> > You're right, this scales badly to long build logs, because of the
> > large number of HTTP requests required. (The size of the logs is so
> > much less of an issue I'm going to ignore that for now.)
>=20
> > For now, I think I'm OK with that, if it's fast enough for
> > not-very-long build logs. From my personal Ick instance, the longest
> > log files are about 14000 lines.
>=20
>   I suspect this depends upon what you're building; for complicated
>  reasons at work I'm creating images which build nginx, php, curl,
>  and similar things from source.  The build output from such jobs
>  is pretty volumous.  At home I tend to build linux-kernels which
>  also have a good few thousand lines of output.

Yup. My personal projects are fairly small and simple. I do want to
build Ick to be able to handle quite large projects, though. The log
file handling will be part of that.

I think we have a sufficiently good design for now, and can progress
with making the Ick controller not have any local, persistent state,
and to use Muck for persistence instead.

>   The stuff in the middle I don't care about so much very often.
>  I'd not suggest that having a ring-buffer was a good idea, but if
>  it were possible to keep track of the last update that might be
>  a compromise worth making.

Getting the tail of the current log, say, the last 100 lines, should
be pretty fast.

>   I suspect when everything is done the build-log belongs with the
>  output artifacts, in the object-store.  That might be personal
>  bias at play of course.

I suspect you're right. If nothing else, the log from one build is
quite similar to the log of the next build, most of the time, and
having a way to store them that takes care of the similarities to
avoid using so much space may well make it worthwhile to have a
special "log service" of some sort. We'll see. That's a "maybe in the
future" idea, not something that's worth doing now, I think.

--=20
I want to build worthwhile things that might last. --joeyh

--orO6xySwJI16pVnm
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAlxy250ACgkQbC+mFux6
IDGBrQ/+LTrnc+MelX8UnRvZJRAH6wbWqoQPirWTTMJ/fIIT3uvyL6Lo4cdUvNnG
MPLGujkVLR4oaFJfCBrhem1MR/Rlsm8Jty8/yZKQr81a1hlq8JViY5X8cTr3x/u8
9lCXLBJAdLolJUvL9X2TsMDG1WVUAezmmR5UfN5XqJP8+RUIk3K7PZhyJ8gDUxH1
A6nY+Kb3oOzW9LP1Ykfvt/b0cCT7ZcJUUgDzm2EysTdScIaXXgdp/O0j4HwnVuvJ
OBDn/SouaMVu8DQvGC7XmVighVkpmMYjKgfes5/Fp/3X1X05fRY3yJ/KN6M1Nc9B
1KGj9f9vTawWpmp+ODtFRwJNsmUf3iQG/F5SCt++pCoE4EEaZAj/rsd+2QXYNbTE
/pxjoDE0mnkIVhoEin7MsXa5BBeu6zHGmIMPQiiLmv6ka6+5BPJFwxyQYgtbyAwl
hZIBtCv8ql9RRqBCuLVQ/m2p68cax1Kggyo8wP3L1QPFTsrkoLp2HTNhhl02A5+w
MHfrWNCF7Hj150FyAPtzHk3G9E5npnTu/EfGikpQac31UJUiZtL04lt6rC83jWtC
bVW76omkODG3zhNObiJfbpSDZeEKd2h5DVU51y9RrZwFiZea0VL4XXuWK7rcj+7m
4KItv1chudeECPnoPLJEAggDn+VLM5Wg0ReL4bJpU75qRQp2w5Q=
=GXCW
-----END PGP SIGNATURE-----

--orO6xySwJI16pVnm--


--===============0442428648705839811==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi

--===============0442428648705839811==--