| [Inclusions of `log4j2` in compiled code](#scan_jndimanager_versionspy) | [Calls to `log4j2` in compiled code](#scan_log4j_calls_jarpy) | [Calls to `log4j2` in source code](#scan_log4j_calls_srcpy) |
| [Sanity check for env mitigations](#env_verifyjar) | [Applicability of CVE-2021-45046](#scan_cve_2021_45046_config) | [Xray wrapper for Log4Shell](#log4shell_xray_wrapper) |
10
-
| [Automatically patch container images in Artifactory](patch_rt_container_registry_repos/python/README.md) | | |
1
+
# commons-text-tools
11
2
12
3
### Overview
13
4
14
-
CVE-2021-44228poses a serious threat to a wide range of Java-based applications. The important questions a developer may ask in this context are:
15
-
16
-
### 1. Does my code include `log4j2`?
17
-
18
-
Does the released code include `log4j2`? Which version of the library is included there? Answering these questions may not be immediate due to two factors:
19
-
20
-
1) Transitive dependencies: while `log4j2` may not be in the direct dependency list of the project, it may be used indirectly by some other dependency.
5
+
CVE-2022-33980maypose a serious threat to a wide range of Java-based applications. The important questions a developer may ask in this context are:
21
6
22
-
2) The code of this library may not appear directly as a separate file (i.e., `log4j2-core-2.xx.0.jar`), but rather be bundled in some other code jar file.
23
-
24
-
JFrog is releasing a tool to help resolve this problem: [`scan_log4j_versions`](#scan_log4j_versionspy). The tool looks for the **class code** of `JndiManager` and `JndiLookup` classes **(regardless of containing `.jar` file names and content of `pom.xml` files)**, and attempts to fingerprint the versions of the objects to report whether the included version of `log4j2` is vulnerable. Both Python and Java implementations are included.
25
-
26
-
### 2. Where does my code use `log4j2`?
27
-
28
-
The question is relevant for the cases where the developer would like to verify if the calls to log4j2 in the codebase may pass potentially attacker-controlled data. While the safest way to fix the vulnerability, as discussed in the advisories, is to apply the appropriate patches and global flags, controlling for and verifying the potential impact under assumption of unpatched `log4j2` may be valuable in many situations. In order to address this problem JFrog is releasing two scripts:
7
+
### 1. Does my code include `commons-text`? Which versions?
29
8
30
-
1. [`scan_log4j2_calls_src.py`](#scan_log4j_calls_srcpy), which locates calls to log4j2 logging functions (info, log, error etc.) with non-constant arguments in *.java source files* and reports the findings on the level of source file and line
31
-
2. [`scan_log4j2_calls_jar.py`](#scan_log4j_calls_jarpy), which locates the calls to logging functions in *compiled .jar*s, and reports the findings as class name and method names in which each call appears.
9
+
Does the released code include `commons-text`? Which version of the library is included there? Answering these questions may not be immediate due to two factors:
32
10
33
-
### 3. Am I configuring this correctly?
11
+
1) Transitive dependencies: while `commons-text` may not be in the direct dependency list of the project, it may be used indirectly by some other dependency.
34
12
35
-
Due to the high risk associated with the vulnerability, developers relying on mitigations may want to double check that the environment was indeed configured correctly (which Java runtime actually runs the application? Were environment and command line flags set correctly?). In order to simplify this sanity check, JFrog is releasing a few tools. The tools are intended to run in the same environment as a production application.
13
+
2) The code of this library may not appear directly as a separate file, but rather be bundled in some other code jar file.
36
14
37
-
* [env_verify.jar](#env_verifyjar) will validate the proper application of mitigations against CVE-2021-44228.
38
-
* [scan_cve_2021_45046_config](#scan_cve_2021_45046_config) will validate the `log4j2` configuration does not allow for exploitation of CVE-2021-45046.
15
+
JFrog is releasing a tool to help resolve this problem: [`scan_commons_text_versions`](#scan_commons_text_versionspy). The tool looks for the **class code** of `StringLookupFactory` **(regardless of containing `.jar` file names and content of `pom.xml` files)**, and attempts to fingerprint the versions of the objects to report whether the included version of `commons-text` is vulnerable.
39
16
40
-
------
17
+
### 2. Does my code use vulnerable `commons-text` functions?
41
18
42
-
## Usage instructions
19
+
The question is relevant for the cases where the developer would like to verify if the calls to `commons-text` in the codebase may pass potentially attacker-controlled data. While the safest way to fix the vulnerability, as discussed in the advisories, is to apply the appropriate patches, controlling for and verifying the potential impact under assumption of unpatched `commons-text` may be valuable in many situations.
43
20
44
-
### `scan_log4j_versions.py`
21
+
[`scan_commons_text_calls_jar.py`](#scan_commons_text_calls_jarpy), which locates the calls to the vulnerable functions in *compiled .jar*s, and reports the findings as class name and method names in which each call appears.
45
22
46
-
The tool requires Python 3, without additional dependencies.
The tool will scan `root_folder` recursively for `.jar` and `.war` files; in each located file the tool looks for a `*log4j/core/net/JndiManager.class`and`*log4j/core/lookup/JndiLookup.class` (recursively in each `.jar` file). If at least one of the classes is found, the tool attempts to fingerprint its version (including some variations found in patches and backport patches) in order to report whether the code is vulnerable.
31
+
The tool will scan `root_folder` recursively for `.jar` and `.war` files; in each located file the tool looks for a `StringLookupFactory.class` (recursively in each `.jar` file). If at least one of the classes is found, the tool attempts to fingerprint its version (including some variations found in patches and backport patches) in order to report whether the code is vulnerable.
61
32
62
-
With `-quiet` flag, only log4jversion conclusions are printed out, and other messages (files not found/ archives failed to open/ password protected archives) are muted.
33
+
With `-quiet` flag, only version conclusions are printed out, and other messages (files not found/ archives failed to open/ password protected archives) are muted.
63
34
64
35
Folders appearing after `-exclude` (optional) are skipped.
To reiterate, the results depend on the code of the classes rather than file names and the metadata. Files where both `JndiManager` and `JndiLookup` classes are not present (and hence are not vulnerable to CVE-2021-44228), like `log4j-1.x.xx.jar`, or `log4j-api-2.xx.x.jar`, do not appear in the results. Otherwise, vulnerability status and estimated version/patch status are displayed. When the versions of the two classes follow a pattern not accounted for, `inconsistent` is reported; this result should be investigated further.
Supported archive extensions: jar, war, ear, sar, par, zip.
77
-
78
37
------
79
38
80
-
### `scan_log4j_versions.jar`
81
-
82
-
Compiled jar can be downloaded from [here](https://releases.jfrog.io/artifactory/log4j-tools/0.0.11/scan_log4j_versions.jar) or [compiled](#compiling-scan_log4j_versionsjar-from-source) from source.
83
-
84
-
The tool requires java runtime, without additional dependencies.
The operation and displayed results are equivalent to the [Python version](#scan_log4j_versionspy).
93
-
94
-
------
95
-
96
-
### `scan_log4j_calls_jar.py`
39
+
### `scan_commons_text_calls_jar.py`
97
40
98
41
The tool requires python 3 and the following 3rd party libraries: `jawa`, `tqdm`, `easyargs`, `colorama`
99
42
skipped 8 lines
108
51
The default use case:
109
52
110
53
```
111
-
python scan_log4j_calls_jar.py root-folder
54
+
python scan_commons_text_calls_jar.py root-folder
112
55
```
113
56
114
-
will recursively scan all `.jar` files in `root-folder`, for each printing out locations (class name and method name) of calls to `info`/`warn`/`error`/`log`/`debug`/`trace`/`fatal` methods of `log4j2.Logger`.
57
+
will recursively scan all `.jar` files in `root-folder`, for each printing out locations (class name and method name) of calls to `lookup`/`replace`/`replaceIn` methods of `StringSubstitutor`/`StringLookup`.
115
58
116
59
The tool may be configured for additional use cases using the following command line flags.
| `--class_regex` | .*log4j/Logger | Regular expression for required class name |
121
-
| `--method_regex` | [^1] | Regular expression for required method name |
122
-
| `--quickmatch_string` | log4j | Pre-condition for file analysis: .jar files not containing the specified string will be ignored |
63
+
| `--class_regex` | `(.*StringSubstitutor|.*StringLookup)` | Regular expression for required class name |
64
+
| `--method_regex` | `(lookup|replace|replaceIn)` | Regular expression for required method name |
65
+
| `--quickmatch_string` | `(StringLookup|StringSubstitutor)` | Pre-condition for file analysis: .jar files not containing the specified regex will be ignored |
123
66
| `--class_existence` | Not set | When not set, look for calls to class::method as specified by regexes. When set, `--method_regex` is ignored, and the tool will look for *existence* of classes specified by `--class_regex` in the jar. |
124
67
| `--no_quickmatch` | Not set | When set, the value of `--quickmatch_string` is ignored and all jar files are analyzed |
125
-
| `--caller_block` | .*org/apache/logging | If caller class matches this regex, it will *not* be displayed |
The tool requires python 3 and the following 3rd party libraries: `javalang`, `tqdm`, `easyargs`, `colorama`
143
-
144
-
##### Dependencies installation
145
-
146
-
```
147
-
pip install -r requirements.txt
148
-
```
149
-
150
-
##### Usage
151
-
152
-
The default use case:
153
-
154
-
```
155
-
python scan_log4j_calls_src.py root-folder
156
-
```
157
-
158
-
will recursively scan all `.java` files in `root-folder`, for each printing out the locations (file name and corresponding code lines) of calls to `log4j2` logging methods.
159
-
160
-
The tool may be configured for additional use cases using the following command line flags:
Compiled jar can be downloaded from [here](https://releases.jfrog.io/artifactory/log4j-tools/0.0.5/env_verify.jar) or [compiled](#compiling-env_verifyjar-from-source) from source, and does not require additional dependencies.
176
-
177
-
#### Usage
178
-
179
-
The intended use is running the tool in the same setting precisely as the production application. For example, for the original launch line in the start-up script:
Jar version can be [compiled](#compiling-scan_cve_2021_45046_configjar-from-source) from source or downloaded from [here](https://releases.jfrog.io/artifactory/log4j-tools/0.0.8/scan_cve_2021_45046_config.jar).
Will recursively scan `root-folder` and all archive files in it, looking for probable log4j configuration files (`xml`, `yml`, `properties`,`json`), in each looking for [configuration options](https://jfrog.com/blog/log4shell-0-day-vulnerability-all-you-need-to-know/#appendix-c) which may enable an attacker to exploit CVE-2021-45046.
224
-
225
-
Please note that an "applicable" result only means that the configuration **may** be problematic and should be inspected.
226
-
227
-
A "non-applicable" result is more conclusive, and means the configuration does not contain even the basic (publicly known) options for the exploitation of CVE-2021-45046.
228
-
229
-
------
230
-
231
-
### `log4shell_xray_wrapper`
232
-
233
-
##### Dependencies
68
+
| `--caller_block` | .*org/apache/commons/text | If caller class matches this regex, it will *not* be displayed |
234
69
235
-
Python version requires installing dependencies:
236
70
237
-
```bash
238
-
pip install -r requirements.txt
239
-
```
240
71
241
-
In addition, the following tools must be available in your `PATH`:
242
-
243
-
* [JFrog CLI](https://www.jfrog.com/confluence/display/CLI/JFrog+CLI#JFrogCLI-Downloadandinstallation) 2.6.2 or later (either `jfrog` or `jf`) - [configured](https://www.jfrog.com/confluence/display/CLI/JFrog+CLI#JFrogCLI-JFrogPlatformConfiguration) with an "Xray URL"
244
-
* Either [maven](https://maven.apache.org/download.cgi) or [gradle](https://gradle.org/install/) (according to the project you are planning to scan)
245
-
246
-
247
-
248
-
##### Usage
249
-
250
-
Jar version can be [compiled](#compiling-log4shell_xray_wrapperjar-from-source) from source or downloaded from [here](https://releases.jfrog.io/artifactory/log4j-tools/0.0.12/log4shell_xray_wrapper-all.jar).
The tool looks for Maven and Gradle projects , either directly at `target_dir` or (if `--recurse` is specified) in any child directory of `target_dir`.
263
-
264
-
Any detected project will be scanned using Xray (via the JFrog CLI), and results will be filtered to show only the Log4Shell vulnerabilities:
265
-
266
-
* CVE-2021-44228
267
-
* CVE-2021-45046
268
-
* CVE-2021-45105
269
-
270
-
271
-
272
-
------
273
-
274
-
### Compiling `scan_log4j_versions.jar` from source
275
-
276
-
```
277
-
cd scan_log4j_versions/java
278
-
gradle build
279
-
cp build/libs/scan_log4j_versions.jar ..
280
-
```
281
-
282
-
------
283
-
284
-
### Compiling `env_verify.jar` from source
285
-
286
-
```
287
-
cd env_verify/java
288
-
gradle build
289
-
cp build/libs/env_verify.jar ..
290
-
```
291
-
292
-
------
293
-
294
-
### Compiling `scan_cve_2021_45046_config.jar` from source
295
-
296
-
```
297
-
cd scan_cve_2021_45046_config/java
298
-
gradle build
299
-
cp build/libs/scan_cve_2021_45046_config.jar ..
300
-
```
301
-
302
-
------
303
-
304
-
### Compiling `log4shell_xray_wrapper.jar` from source
Ilya Khivrich
committed
2 years ago
1 parent da46a48e