BY MY HANDS. Neural Network Training

Modern neural networks, such as Stable Diffusion XL, are capable of generating images from textual descriptions. However, basic models aim for a general result and do not account for an individual’s artistic style. If an artist wishes to utilize AI while preserving their unique manner—such as line quality, color palette, or textural specifics—standard tools do not provide this capability.

Adapt the pre-trained SDXL model so that it can generate images in my personal artistic style while maintaining control over the content via text prompts.

To build a robust foundation for learning, I selected 23 of my drawings that best reflect my characteristic traits in drawing: compositional structure, color palette, line texture, and overall aesthetic.

This quantity proved sufficient—LoRA does not require thousands of examples like training from scratch. What matters more is quality and diversity: the dataset includes works with different subjects but a unified style, allowing the model to learn to generalize stylistic features rather than memorize specific images.

Using these two illustrations as examples, I would detail the specific characteristics of my signature style that the model I trained must preserve.

Key Narratives:

1. Vivid plasticity of lines
2. Deliberate distortion of natural poses to create dynamism
3. Contrast in light and shadow
4. Imperfection—intentionally maintaining the feeling of a sketch, swift hatching, and «drawing in the flow»
5. Active manipulation of spatial perspective

Details and Signature Elements:

1. Emphasized eye fatigue, a languid gaze
2. Tousled hair with vibrant highlights
3. Colored line art

To ensure they could be worked with without technical difficulties, I implemented a resize function directly in the code when extracting the archive with the source images.

Training was conducted using the DreamBooth + LoRA method.

Preparation: I selected a small collection of my original drawings that most vividly reflected the authorial style the final model needed to adopt.

Tagging: I added a unique hhellonedd style tag to each caption. This functions like a «label» that allows the neural network to understand exactly which set of visuals must be associated with a specific style name.

Training: I initiated a process where a large neural network, SDXL, attempted to reproduce my drawings. I used the (8-bit Adam) algorithm, which allowed for efficient and rapid model training. Ultimately, I obtained a small file with «weights» (LoRA), which overlays the base model as a stylistic filter.

It is also important to mention the settings and details that helped me improve the fidelity to the style presented in the original images:

VAE (Encoder):

I integrated a specialized corrected component (fp16-fix), which is responsible for ensuring that the final images do not contain minor artifacts or «broken» pixels.

Memory and Optimization:

Since SDXL is a very resource-intensive neural network, I utilized CPU Offloading and VAE Tiling. This enabled me to generate high-resolution images (1024×1024) using the free version of Google Colab.

Style Control:

I set the parameter num_inference_steps=25–30, which strikes a perfect balance: the neural network has enough steps to render the style details without overcomplicating the image.

The Difference Between 250 and 500 Steps:

250 Steps: At this stage, the neural network has grasped the general characteristics of the style (color palette, primary forms), but the images remain softer—sometimes appearing «watercolor-like» or generalized. This works well for more abstract concepts.

500 Steps: Here, the model has internalized the style much more deeply. The lines become sharper, and characteristic details (hatching, contour specifics) emerge more vividly. The images become more recognizable and specific to this particular artistic touch.

For a detailed analysis of my work’s results, I would refer to the very first outcome I received, which helped me understand how I should ultimately refine the code.

1. drawing in hellonedd style, girl playing badminton

The first result didn’t seem the most successful, so for subsequent images, I added the pipe.vae.enable_tiling () method.

Without this function in the first result, the model «compressed» the entire image, causing the subtle characteristics of my style—such as the quality of the line work and the nuances of color transitions—to be averaged out and lost. Once tiling was enabled, the model began processing the image in sections, preserving fine details in each area. This allowed the LoRA to convey my signature style more accurately, especially in the textures and minute elements that define its recognizability.

2. Drawing in hellonedd style, boy running

After activating this feature, I immediately received a satisfying result.

It vividly reflects how I work with the plasticity of poses, the overall sketchiness of the image, the emphasis on shadows in eye sockets, and even my penchant for dynamic folds in clothing.

3. drawing in hhellonedd style, girl holding the sun over her head with her hands portrait

4. drawing in hhellonedd style, a girl sitting in a water, aerial view

Using these two generations as an example, I would like to examine how the model independently handles the stylization of elements—the sun and the water—that are absent from any of the source images.

As someone who directly created this work with textures and lines, I can say that the trained model successfully adapted these phenomena to the style: the sun rays are rendered with the dynamic quality matching my personal technique, and the water is depicted with distortions of its real physical properties, yet aligned with my principles of contrast in light and shadow.

5. drawing in hellonedd style, boy in big scarf standing under the snowfall at night

I think this generation clearly reflects my recurring fondness for oversized clothing and the baggy silhouette it creates, a trait visible in many of the dataset’s drawings.

Perhaps this tendency is more indicative of my personal aesthetic preferences in life, and I don’t intend it as a mandatory constant in my drawing style, but the fact that the model has independently recognized and integrated this frequent detail in some generations seems quite fascinating to me.

6. drawing in hhellonedd style, portrait of a boy with vivid blue contrasting light

7. drawing in hhellonedd style, a boy and a girl holding hands and walking, colored drawing

These two generations are the most striking examples for analyzing the handling of chiaroscuro.

It seems to me that the AI has captured the character of my approach to light contrast and has grasped the hyperbolization of skin highlights in a way that is sufficiently recognizable to my style.

In the presented generations, it is evident how LoRA helped train SDXL not just to copy individual elements, but to flexibly apply my style to diverse subjects, creating interesting variations within a unified aesthetic. To elaborate on this resulting adaptability, I would highlight the following points for consideration: 1. Dynamics of Poses and Lines

The completed series of generations confirms that the model successfully interpreted dynamics in varied ways: in images where the characters’ poses were static, it emphasized dynamism through line plasticity; whereas, when dealing with active object movement, in line with my signature style, it intentionally and interestingly distorted and hyperbolized the physics of the human body.

2. Fundamental Handling of Hatching

In the sketched versions, the hatching remains quick, textural, and visibly unfinished. Even when the overall rendering in some generations became denser, the characteristic texture of my hatching was still preserved. This suggests that the model grasped the range I provided—from a rough sketch to a more finished piece—and varies it according to the narrative.

3. Stylizing Elements Absent in the Dataset

The resulting generative model clearly shows the potential to move beyond the rote repetition of dataset fragments and transition into the adaptive application of stylistic traits. It successfully managed to stylize new elements that did not appear in the original drawings. This proves that LoRA learned the principles of my style rather than simply memorizing specific images.

4. Accurate Definition of Fine Details

From one generation to the next, certain details remained appropriately consistent. The trained model successfully fixed the importance of highlighting specular reflections on strands of hair, the dishevelment of hairstyles, defining under-eye bags, and the bagginess of the clothing.

Following a thorough analysis, I can conclude that the task of «digitizing» my vision of this world has been successfully completed, and the resulting model holds significant potential for further use due to the proper analysis algorithm and appropriate stylistic framing.