mode.utils.logging
¶
Logging utilities.
- class mode.utils.logging.CompositeLogger(logger: Logger, formatter: Optional[Callable[[...], str]] = None)¶
Composite logger for classes.
The class can be used as both mixin and composite, and may also define a
.formatter
attribute which will reformat any log messages sent.Service uses this to add logging methods:
class Service(ServiceT): log: CompositeLogger def __init__(self): self.log = CompositeLogger( logger=self.logger, formatter=self._format_log, ) def _format_log(self, severity: int, message: str, *args: Any, **kwargs: Any) -> str: return (f'[^{"-" * (self.beacon.depth - 1)}' f'{self.shortlabel}]: {message}')
This means those defining a service may also use it to log:
>>> service.log.info('Something happened')
and when logging additional information about the service is automatically included.
- format(severity: int, message: str, *args: Any, **kwargs: Any) str ¶
- log(severity: int, message: str, *args: Any, **kwargs: Any) None ¶
- logger: Logger¶
- class mode.utils.logging.DefaultFormatter(fmt=None, datefmt=None, style='%', validate=True, *, defaults=None)¶
Default formatter adds support for extra data.
- format(record: LogRecord) str ¶
Format the specified record as text.
The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.
- class mode.utils.logging.ExtensionFormatter(stream: Optional[IO] = None, **kwargs: Any)¶
Formatter that can register callbacks to format args.
Extends :pypi:`colorlog`.
- format(record: LogRecord) str ¶
Format the specified record as text.
The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.
- format_arg(arg: Any, record: LogRecord) Any ¶
- class mode.utils.logging.FileLogProxy(logger: Logger, *, severity: Optional[Union[int, str]] = None)¶
File-like object that forwards data to logger.
- property buffer: BinaryIO¶
- close() None ¶
- property closed: bool¶
- property encoding: str¶
- property errors: str | None¶
- fileno() int ¶
- flush() None ¶
- isatty() bool ¶
- line_buffering() bool ¶
- property mode: str¶
- property name: str¶
- property newlines: bool¶
- read(n: int = -1) AnyStr ¶
- readable() bool ¶
- readline(limit: int = -1) AnyStr ¶
- readlines(hint: int = -1) list[AnyStr] ¶
- seek(offset: int, whence: int = 0) int ¶
- seekable() bool ¶
- severity: int = 30¶
- tell() int ¶
- truncate(size: Optional[int] = None) int ¶
- writable() bool ¶
- write(s: AnyStr) int ¶
- writelines(lines: Iterable[str]) None ¶
- class mode.utils.logging.Logwrapped(obj: Any, logger: Optional[Any] = None, severity: Union[int, str] = 30, ident: str = '')¶
Wrap all object methods, to log on call.
- ident: str¶
- logger: Any¶
- obj: Any¶
- severity: int¶
- mode.utils.logging.cry(file: IO, *, sep1: str = '=', sep2: str = '-', sep3: str = '~', seplen: int = 49) None ¶
Return stack-trace of all active threads.
See also
Taken from https://gist.github.com/737056.
- class mode.utils.logging.flight_recorder(logger: Logger, *, timeout: Union[timedelta, int, float, str], loop: Optional[AbstractEventLoop] = None)¶
Flight Recorder context for use with
with
statement.This is a logging utility to log stuff only when something times out.
For example if you have a background thread that is sometimes hanging:
class RedisCache(mode.Service): @mode.timer(1.0) def _background_refresh(self) -> None: self._users = await self.redis_client.get(USER_KEY) self._posts = await self.redis_client.get(POSTS_KEY)
You want to figure out on what line this is hanging, but logging all the time will provide way too much output, and will even change how fast the program runs and that can mask race conditions, so that they never happen.
Use the flight recorder to save the logs and only log when it times out:
logger = mode.get_logger(__name__) class RedisCache(mode.Service): @mode.timer(1.0) def _background_refresh(self) -> None: with mode.flight_recorder(logger, timeout=10.0) as on_timeout: on_timeout.info(f'+redis_client.get({USER_KEY!r})') await self.redis_client.get(USER_KEY) on_timeout.info(f'-redis_client.get({USER_KEY!r})') on_timeout.info(f'+redis_client.get({POSTS_KEY!r})') await self.redis_client.get(POSTS_KEY) on_timeout.info(f'-redis_client.get({POSTS_KEY!r})')
If the body of this
with
statement completes before the timeout, the logs are forgotten about and never emitted – if it takes more than ten seconds to complete, we will see these messages in the log:[2018-04-19 09:43:55,877: WARNING]: Warning: Task timed out! [2018-04-19 09:43:55,878: WARNING]: Please make sure it is hanging before restarting. [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (started at Thu Apr 19 09:43:45 2018) Replaying logs... [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:45 2018) +redis_client.get('user') [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:49 2018) -redis_client.get('user') [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] (Thu Apr 19 09:43:46 2018) +redis_client.get('posts') [2018-04-19 09:43:55,878: INFO]: [Flight Recorder-1] -End of log-
Now we know this
redis_client.get
call can take too long to complete, and should consider adding a timeout to it.- activate() None ¶
- blush() None ¶
- cancel() None ¶
- enabled_by: _asyncio.Task | None¶
- extra_context: dict[str, Any]¶
- flush_logs(ident: Optional[str] = None) None ¶
- log(severity: int, message: str, *args: Any, **kwargs: Any) None ¶
- logger: Logger¶
- loop: AbstractEventLoop¶
- started_at_date: str | None¶
- timeout: float¶
- wrap(severity: int, obj: Any) Logwrapped ¶
- wrap_debug(obj: Any) Logwrapped ¶
- wrap_error(obj: Any) Logwrapped ¶
- wrap_info(obj: Any) Logwrapped ¶
- wrap_warn(obj: Any) Logwrapped ¶
- mode.utils.logging.formatter(fun: Callable[[Any], Any]) Callable[[Any], Any] ¶
Register formatter for logging positional args.
- mode.utils.logging.formatter2(fun: Callable[[Any, LogRecord], Any]) Callable[[Any, LogRecord], Any] ¶
Register formatter for logging positional args.
Like
formatter()
but the handler accepts two arguments instead of one:(arg, logrecord)
.Passing the log record as additional argument expands the capabilities of the formatter to do things such as read the log message.
- mode.utils.logging.get_logger(name: str) Logger ¶
Get logger by name.
- mode.utils.logging.level_name(log_level: int | str) str ¶
- mode.utils.logging.level_name(log_level: int) str
- mode.utils.logging.level_name(log_level: str) str
Convert log level to number.
- mode.utils.logging.level_number(log_level: int | str) int ¶
- mode.utils.logging.level_number(log_level: int) int
- mode.utils.logging.level_number(log_level: str) int
Convert log level number to name.
- mode.utils.logging.redirect_stdouts(logger: ~logging.Logger = <Logger mode.redirect (WARNING)>, *, severity: ~typing.Optional[~typing.Union[int, str]] = None, stdout: bool = True, stderr: bool = True) Iterator[FileLogProxy] ¶
Redirect
sys.stdout
andsys.stdout
to logger.
- mode.utils.logging.setup_logging(*, log_level: Optional[Union[int, str]] = None, log_file: Optional[Union[PathLike, str, IO]] = None, log_handlers: Optional[Iterable[Handler]] = None, logging_config: Optional[dict] = None) int ¶
Configure logging subsystem.