Configuring logging
See details in Apache Spark documentation.
The table below could help you in selection of appropriate log level:
Level | Description |
---|---|
OFF | The highest possible rank and is intended to turn off logging. |
FATAL | Designates very severe error events that will presumably lead the application to abort. |
ERROR | Designates error events that might still allow the application to continue running. |
WARN | Designates potentially harmful situations. |
INFO | Designates informational messages that highlight the progress of the application at coarse-grained level. |
DEBUG | Designates fine-grained informational events that are most useful to debug an application. |
TRACE | Designates finer-grained informational events than the DEBUG. |
ALL | All levels including custom levels. |