It turns out that Stable Diffusion is not only good at creating Anime. It is also an excellent tool for photography. I have recently been experimenting with black and white photos, specifically in a photo-journalism style. Some of my favourite images were produced by photo-journalists covering apartheid-era marches, riots, and fighting.
This picture by Sam Nzima Mbuyisa Makhubo carrying Hector Pietersen who was shot by the South African police is perhaps the most iconic photo taken during that time.
Despite over 100 years of colour photography, black and white photos continue to capture not only the scene, but also the mood.
Those apartheid images carry the weight of history in a way that would be difficult/impossible to replicate artificially, but it’s still possible to capture something that evokes an emotional response.
They can also remind us of a time that never was.
Prompt Engineering
In this short blog, I’m going to explore various modifiers and how they affect your images.
Before we start, a caveat. For all the experiments below, I am using a single base prompt. The results presented may be completely different to your findings if you use another prompt or model.
Let’s start with a simple prompt which we can experiment with:
A photograph portrait of an elderly woman
If you want to follow along, I am using the the base 1.5 stable diffusion model with dimensions 512x704 and seed 1418630671. Unless otherwise specified, I am using the Euler-A sampler, and a CFG value of either 7 or 9.
Notice that I didn’t mention black and white in my prompt although the image is infact black and white. I think this is purely coincidental. Another seed might have generated a colour image.
On that topic, what is the best way to communicate to Stable Diffusion that we want a black and white image.
Note, I am using the Dynamic Prompt extension for the Automatic1111 webui to create my images. {|B&W|Black and White|Monochrome} is the syntax used by the extension to create four different images. Each image uses a different modifier:
- A photograph portrait of an elderly woman
- A B&W photograph of an elderly woman
- A Black and White photograph of an elderly woman
- A Monochrome photograph of an elderly woman
I quite like Black and White because of the deep contrast which accentuates her wrinkles and deepens her eyes. You might argue that it’s a little underexposed. B&W and Monochrome also have a strong contrast. I find the images that doesn’t use a modifier at all to be a little dull in comparison.
Step count
There are many dials and knob you can adjust when creating your image. Step count is one of them and determines how long the algorithm runs for. Too few steps and the image is barely formed. Too many steps and your images take a long time to generate with little change to the final image quality.
10 steps is obviously too few. The image isn’t even in black and white. I don’t see much difference in any of the other images. In that case, for the sake of saving time, the remaining experiments will use 20 steps.
Classifier Free Guidance Scale
CFG scale is often described as a how closely you want Stable Diffusion to follow the prompt. The lower the number, the more creative it can get. Too high a number can result in distorted images. You can read more about it here.
CFG 1 looks barely formed. You can recognise the shape of a person but not much more
CFG 2 is a low contrast image
CFG 4 is starting to look good. You might want to stop here.
CFG 6–12 have increasing degrees of contrast.
CFG 16 and 20 have gone off script, producing colour images. Image grain starts to become more prominent.
CFG 24 gives an interesting low-res, very grainy effect.
I’m going to choose CFG 9 for the rest of the images in this blog.
Denoising Sampler
Samplers are the most mathematical part of the diffusion process, and the least well-understood. Below are some links for various explanations of how they work and what they do. I chose a handful of samplers to test. DDIM has the lowest contrast and the result isn’t very sharp. PLMS Karras and Euler are similar, but a little sharper. Heun and the DPM family of samplers produce high contrast, very sharp images. I’m sticking to Euler-A for the rest of this blog, but it’s worth experimenting with the others since the seem to have a very strong impact on the final image.
https://www.artstation.com/blogs/kaddoura/pBPo/stable-diffusion-samplers
Various Camera modifiers
Here I’m testing various camera modifiers in order to get a sense of what they do. I have previously had much success with CCTV. In this example, it too underexposed. Kodak Retina II and Nikon produce pleasing images as well. Not what I’m looking for though. I want darker lines and deeper shadows.
Film Type
I’ve tried dozens of films from a number of manufacturers such as Rollei and Loma with minimal difference. Here are some images created using Kodak’s line-up of black and white film. The films are described as follows:
Tri-X 320: Lower contrast profile than Tri-X 400. Smoother grain structure.
Tri-X 400: Classic. Grainy
T-Max 100: Similar to T-Max 400
T-Max 400: Contemporary. Fine grain, Smooth
T-Max 3200: Low-light. salt and pepper look of high speed film
I’m not seeing much grain in the images apart from T-Max 3200. Below is a larger image to show you the grain.
ISO
In film, ISO refers to how sensitive it is to light. The higher the ISO, the more sensitive it is to light, the shorter the amount of time that the camera’s shutter needs to be open to collect light.
You would likely want lower iso films in broad daylight whereas, a photo of a 100m sprint would need a high iso-valued film in order to avoid needing to keep the shutter open too long resulting in a blurred image as Usain Bolt flies past the camera.
Your mileage may vary but in my experience, ISO has little to no impact on Stable Diffusion. In the examples above, higher ISO values tend to result in darker images rather than lighter ones which a higher sensitivity to light would imply. In other experiments, there was no difference at all.
Aperture
Aperture refers to how much light passes through the camera lens, in the same way that our pupils increase in size in low-light and decrease when we walk outside. A wide aperture can overexpose images in bright environments similar to how we get momentarily blinded when walking from a dark environment into a light one.
Aperture is measure in f-stops. The lower the aperture value, the more light passes through the lens. Aperture also has a secondary purpose. It affects the depth of field of an image. A low-valued aperture produces images with a shallow depth of field where background objects become blurry. Lower apertures are usually used for portraits whereas high apertures are more likely in landscape photograph.
I haven’t experimented much with this modifier but I don’t noticed a strong trend, apart from the change between not using a modifier at all and any aperture value.
Exposure modifiers
I took a slight different approach to testing exposure modifiers. Some Stable Diffusion tools allow for negative prompts. Whereas regular prompts instruct the algorithm what the image should be, negative prompts suggest what the image should not be. In the matrix above, I experimented with adding the terms underexposed and overexposed in both the positive and negative prompts.
The top left image is a base image without modifiers. The image in the second row and first column has the standard prompt “A black and white photograph portrait of an elderly woman” and “overexposed” in the negative prompt. To be read as the image should not be overexposed.
These modifiers definitely affect the final image. Take a look at the last column, i.e. all images must be underexposed. Both row 1 and row 2 are quite dark. In the bottom row of that column, our subject still has dark pools for eyes, but less so than the other two. The middle column represents prompts which included overexposed in the positive prompt. In contrast to the last column, there are very few shadows in these pictures.
Grain
Film can be more or less grainy. This is as a result of silver particles in the film itself. Images can also seem grainier if they were taken in low light are with higher ISO films. They are synonymous with action shots, and can give a raw, gritty feel to your images. Let’s add the “grainy” modifier to our images to see its effect.
The (grainy:number) used here is used by Automatic1111 to apply additional emphasis to that term. We can see very strong grain effects at values 1.6 and 1.8. It’s worth keeping this in mind when trying to get an authentic feel to images to our images.
Miscellaneous
Here are some miscellaneous modifiers that may get you the image you are looking for:
After generating around 450 images for this blog, I’ve settled on this image as my favourite.
While I liked how dramatic some of the high contrast images were, this one seems to keep in that spirit while being less contrived.
Conclusion
I hope this was a useful tour of some modifiers you can use for black and white photos. What we’ve seen is that terms that are important for real-life photography, such as ISO and aperture have little-to-no influence on our images. More important is to check different samplers and CFG values. The terms grainy, overexposed and underexposed are also useful.
As I mentioned at the start, these modifiers may affect your prompts differently so it is worth testing.
The two most valuable tools for this type of batch experimentation is Dynamic Prompts for changing prompts, and XY Plot, which is bundled with Automatic1111, for adjusting settings such as step count, and CFG.
If you find other useful modifiers or techniques to improve the creation of black and white photographs, please leave them in the comments.