What I Read This Season: Fall 2024
Last updated
Last updated
October 2024
What I Read This Season is a new series that I've decided to start in order to both share papers I've read and motivate myself keep up with computer science and cybersecurity literature. I was planning to post this in at the end of fall (around December), but I got a little excited.
Zachary C. Lipton and Jacob Steinhardt
One of the professors I researched with emphasized the importance of rigorous research and shared this paper with the lab. The authors in this paper point out trends in machine learning research that undermine rigorous research and propose ways to combat each of these trends.
I personally found that this paper was thoughtfully written. The authors supported each of the trends they pointed out with specific examples. Their paper critiques were carefully and constructively worded. The authors also took care to criticize either their own work or the work senior researchers rather than of the work of junior researchers. This decision clearly reflects the authors' commitment to improving the rigor of machine learning research while contributing to a supportive and welcoming research community.
You can read the paper here.
By Mark Scanlon, Frank Breitinger, Christopher Hargreaves, Jan-Niclas Hilgert, and John Sheppard
I learned about this paper through the 2023 Forensic 4:cast awards. This paper evaluates how effective ChatGPT-4 is at various digital forensic tasks, from artifact identification to self-directed learning.
I really liked this paper for the breath it provided. I enjoyed the variety of possibilities suggested by the authors, and I think that this paper can give digital forensics professionals new ideas about how to adapt their workflow to include large language models. This paper touched on many interesting applications that I had not even imagined, and I think that each topic opens up doors for further research and exploration. I would love to see more research that builds off of each of the possible applications of ChatGPT for digital forensic investigations mentioned in this paper.
I would love to see the authors elaborate more on the prompts they used, as prompting techniques have been shown to improve the responses of large language models (LLMs) such as ChatGPT-4. To properly evaluate which prompts work better for LLMs in digital forensic applications, future work will likely have to find a way to systematically determine what makes one response better than another. In addition, the paper mentions how ChatGPT-4 has non-deterministic behavior. While this is true, there are ways to manipulate the level of deterministic behavior in ChatGPT models by interacting with these models through their API and setting the temperature parameter. The ChatGPT-4 API was not widely available during this paper's research. I would love to know more about the rational for choosing a newer ChatGPT model without an API over an older ChatGPT model with an API.
You can read the paper here.
â‹… â‹… Thoughts