--- name: "squid2dlf" section: "1" platform: "linux" package: "lire" version: "20020214-7" release: "Debian 3.0 (Woody)" platforms: ["linux"] url: "https://new.carta.tech/squid2dlf" --- # squid2dlf(1) convert squid logs to dlf format ## NAME squid2dlf - convert squid logs to dlf format ## SYNOPSIS **squid2dlf** ## DESCRIPTION squid2dlf expects an access.log log file, as produced by the SQUID Web Proxy Cache ( http://www.squid-cache.org/ ) on it's stdin. These log files are whitespace separated, columns are ``` time elapsed remotehost code/status bytes method URL rfc931 peerstatus/peerhost type ``` , a typical logline looks like e.g. ``` 979992041.366 502 192.168.1.160 TCP_IMS_HIT/304 216 GET http://example.com/home2.html - NONE/- text/html ``` Meaning of fields is: - time A Unix timestamp as UTC seconds with a millisecond resolution. - elapsed The elapsed time considers how many milliseconds the transaction busied the cache. - remotehost The IP address of the requesting instance, the client IP address. - code/status This column is made up of two entries separated by a slash. This column encodes the transaction result. The cache result of the request contains information on the kind of request, how it was satisfied, or in what way it failed. The status part contains the HTTP result codes with some Squid specific extensions. Squid uses a subset of the RFC defined error codes for HTTP. - TCP\_HIT A valid copy of the requested object was in the cache. - TCP\_MISS The requested object was not in the cache. - TCP\_REFRESH\_HIT The requested object was cached but STALE. The IMS query for the object resulted in "304 not modified". - TCP\_REF\_FAIL\_HIT The requested object was cached but STALE. The IMS query failed and the stale object was delivered. - TCP\_REFRESH\_MISS The requested object was cached but STALE. The IMS query returned the new content. - TCP\_CLIENT\_REFRESH\_MISS The client issued a "no-cache" pragma, or some analogous cache control command along with the request. Thus, the cache has to refetch the object. - TCP\_IMS\_HIT The client issued an IMS request for an object which was in the cache and fresh. - TCP\_SWAPFAIL\_MISS The object was believed to be in the cache, but could not be accessed. - TCP\_NEGATIVE\_HIT Request for a negatively cached object, e.g. "404 not found", for which the cache believes to know that it is inaccessible. - TCP\_MEM\_HIT A valid copy of the requested object was in the cache and it was in memory, thus avoiding disk accesses. - TCP\_DENIED Access was denied for this request. - TCP\_OFFLINE\_HIT The requested object was retrieved from the cache during offline mode. The offline mode never validates any object, see offline\_mode in squid.conf file. - UDP\_HIT A valid copy of the requested object was in the cache. - UDP\_MISS The requested object is not in this cache. - UDP\_DENIED Access was denied for this request. - UDP\_INVALID An invalid request was received. - UDP\_MISS\_NOFETCH During "-Y" startup, or during frequent failures, a cache in hit only mode will return either UDP\_HIT or this code. Neighbours will thus only fetch hits. - NONE Seen with errors and cachemgr requests. - bytes The size is the amount of data delivered to the client. Mind that this does not constitute the net object size, as headers are also counted. - method The request method to obtain an object. Methods are: ``` method defined cachabil. meaning --------- ---------- ---------- -------------------- GET HTTP/0.9 possibly object retrieval and simple searches. HEAD HTTP/1.0 possibly metadata retrieval. POST HTTP/1.0 CC or Exp. submit data (to a program). PUT HTTP/1.1 never upload data (e.g. to a file). DELETE HTTP/1.1 never remove resource (e.g. file). TRACE HTTP/1.1 never appl. layer trace of request route. OPTIONS HTTP/1.1 never request available comm. options. CONNECT HTTP/1.1r3 never tunnel SSL connection. ICP_QUERY Squid never used for ICP based exchanges. PURGE Squid never remove object from cache. PROPFIND rfc2518 ? retrieve properties of an object. PROPATCH rfc2518 ? change properties of an object. MKCOL rfc2518 never create a new collection. MOVE rfc2518 never create a duplicate of src in dst. COPY rfc2518 never atomically move src to dst. LOCK rfc2518 never lock an object against modifications. UNLOCK rfc2518 never unlock an object. ``` - URL This column contains the URL requested. Please note that the log file may contain whitespaces for the URI. In the Lire DLF, the URL is split up in proto, host and path. - rfc931 The eigth column may contain the ident lookups for the requesting client. - peerstatus/peerhost results from neigbouring caches. - type The content type of the object as seen in the HTTP reply header. ## VERSION $Id: squid2dlf.in,v 1.6 2002/02/06 22:08:23 flacoste Exp $ ## AUTHORS Joost Bekkers \, Joost van Baal \ ## COPYRIGHT Copyright (C) 2001 Joost Bekkers \ Copyright (C) 2001,2002 Stichting LogReport Foundation LogReport@LogReport.org This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with http://www.gnu.org/copyleft/gpl.html or write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA.