1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
|
Return-Path: <ick-discuss-bounces@ick.liw.fi>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
by pieni.net (Postfix) with ESMTPS id 08BFD414EC
for <distix@pieni.net>; Sun, 24 Feb 2019 18:00:02 +0000 (UTC)
Received: from platypus.pepperfish.net (unknown [10.112.101.20])
by yaffle.pepperfish.net (Postfix) with ESMTP id C59554187F;
Sun, 24 Feb 2019 18:00:01 +0000 (GMT)
Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1gxy4H-0002iI-LD; Sun, 24 Feb 2019 18:00:01 +0000
Received: from koom.pieni.net ([88.99.190.206] helo=pieni.net)
by platypus.pepperfish.net with esmtpsa (Exim 4.80 #2 (Debian))
id 1gxy4G-0002hG-Fj
for <ick-discuss@ick.liw.fi>; Sun, 24 Feb 2019 18:00:00 +0000
Received: from exolobe1.liw.fi (62-78-212-250.bb.dnainternet.fi
[62.78.212.250]) by pieni.net (Postfix) with ESMTPSA id F108B414EC
for <ick-discuss@ick.liw.fi>; Sun, 24 Feb 2019 17:59:59 +0000 (UTC)
Received: from exolobe1.liw.fi (localhost [127.0.0.1])
by exolobe1.liw.fi (Postfix) with ESMTPS id 302F6A83803
for <ick-discuss@ick.liw.fi>; Sun, 24 Feb 2019 19:59:59 +0200 (EET)
Date: Sun, 24 Feb 2019 19:59:58 +0200
From: Lars Wirzenius <liw@liw.fi>
To: ick-discuss@ick.liw.fi
Message-ID: <20190224175958.GC17934@exolobe1.liw.fi>
References: <5e2b278847b76dd0311d5050b3455b12b4dd3077.camel@liw.fi>
<1549688875.1308.1@ssh.steve.org.uk>
<20190216144009.GA5870@exolobe1.liw.fi>
<1550678063.32630.0@ssh.steve.org.uk>
MIME-Version: 1.0
In-Reply-To: <1550678063.32630.0@ssh.steve.org.uk>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Pepperfish-Transaction: beab-34b9-06b8-1f64
X-Pepperfish-Transaction-By: platypus
Subject: Re: Plan for using Muck for the Ick controller
X-BeenThere: ick-discuss@ick.liw.fi
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: discussions about the ick CI system <ick-discuss-ick.liw.fi>
List-Unsubscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/ick-discuss-ick.liw.fi>
List-Post: <mailto:ick-discuss@ick.liw.fi>
List-Help: <mailto:ick-discuss-request@ick.liw.fi?subject=help>
List-Subscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============0442428648705839811=="
Mime-version: 1.0
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi
--===============0442428648705839811==
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature"; boundary="orO6xySwJI16pVnm"
Content-Disposition: inline
--orO6xySwJI16pVnm
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Wed, Feb 20, 2019 at 03:54:23PM +0000, Steve Kemp wrote:
> > (Sorry about taking so long to answer: I was travelling for work.)
>=20
> No worries.
>=20
> > You're right, this scales badly to long build logs, because of the
> > large number of HTTP requests required. (The size of the logs is so
> > much less of an issue I'm going to ignore that for now.)
>=20
> > For now, I think I'm OK with that, if it's fast enough for
> > not-very-long build logs. From my personal Ick instance, the longest
> > log files are about 14000 lines.
>=20
> I suspect this depends upon what you're building; for complicated
> reasons at work I'm creating images which build nginx, php, curl,
> and similar things from source. The build output from such jobs
> is pretty volumous. At home I tend to build linux-kernels which
> also have a good few thousand lines of output.
Yup. My personal projects are fairly small and simple. I do want to
build Ick to be able to handle quite large projects, though. The log
file handling will be part of that.
I think we have a sufficiently good design for now, and can progress
with making the Ick controller not have any local, persistent state,
and to use Muck for persistence instead.
> The stuff in the middle I don't care about so much very often.
> I'd not suggest that having a ring-buffer was a good idea, but if
> it were possible to keep track of the last update that might be
> a compromise worth making.
Getting the tail of the current log, say, the last 100 lines, should
be pretty fast.
> I suspect when everything is done the build-log belongs with the
> output artifacts, in the object-store. That might be personal
> bias at play of course.
I suspect you're right. If nothing else, the log from one build is
quite similar to the log of the next build, most of the time, and
having a way to store them that takes care of the similarities to
avoid using so much space may well make it worthwhile to have a
special "log service" of some sort. We'll see. That's a "maybe in the
future" idea, not something that's worth doing now, I think.
--=20
I want to build worthwhile things that might last. --joeyh
--orO6xySwJI16pVnm
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEETNTnrewG6wEE1EJ3bC+mFux6IDEFAlxy250ACgkQbC+mFux6
IDGBrQ/+LTrnc+MelX8UnRvZJRAH6wbWqoQPirWTTMJ/fIIT3uvyL6Lo4cdUvNnG
MPLGujkVLR4oaFJfCBrhem1MR/Rlsm8Jty8/yZKQr81a1hlq8JViY5X8cTr3x/u8
9lCXLBJAdLolJUvL9X2TsMDG1WVUAezmmR5UfN5XqJP8+RUIk3K7PZhyJ8gDUxH1
A6nY+Kb3oOzW9LP1Ykfvt/b0cCT7ZcJUUgDzm2EysTdScIaXXgdp/O0j4HwnVuvJ
OBDn/SouaMVu8DQvGC7XmVighVkpmMYjKgfes5/Fp/3X1X05fRY3yJ/KN6M1Nc9B
1KGj9f9vTawWpmp+ODtFRwJNsmUf3iQG/F5SCt++pCoE4EEaZAj/rsd+2QXYNbTE
/pxjoDE0mnkIVhoEin7MsXa5BBeu6zHGmIMPQiiLmv6ka6+5BPJFwxyQYgtbyAwl
hZIBtCv8ql9RRqBCuLVQ/m2p68cax1Kggyo8wP3L1QPFTsrkoLp2HTNhhl02A5+w
MHfrWNCF7Hj150FyAPtzHk3G9E5npnTu/EfGikpQac31UJUiZtL04lt6rC83jWtC
bVW76omkODG3zhNObiJfbpSDZeEKd2h5DVU51y9RrZwFiZea0VL4XXuWK7rcj+7m
4KItv1chudeECPnoPLJEAggDn+VLM5Wg0ReL4bJpU75qRQp2w5Q=
=GXCW
-----END PGP SIGNATURE-----
--orO6xySwJI16pVnm--
--===============0442428648705839811==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi
--===============0442428648705839811==--
|
