See details in Apache Spark documentation.
The table below could help you in selection of appropriate log level:
|The highest possible rank and is intended to turn off logging.
|Designates very severe error events that will presumably lead the application to abort.
|Designates error events that might still allow the application to continue running.
|Designates potentially harmful situations.
|Designates informational messages that highlight the progress of the application at coarse-grained level.
|Designates fine-grained informational events that are most useful to debug an application.
|Designates finer-grained informational events than the DEBUG.
|All levels including custom levels.