Update News Presentation Markup for Games

šŸ”– game-development ā²ļø 4 minutes to read

During the development of Estranged: The Departure, I needed a way to bring update news posts into the main game menu, to allow players at a glance to see "what's new". Below is the final result - responsive, scrollable, native news UI which optionally links out to a browser.

I initially did this using the only sane way - an embedded web browser. However, this wasn't very performant and required an extra process sitting in the background of the game. It also caused a few shutdown crashes at the time (though I imagine newer versions of Unreal Engine don't have this problem).

I decided I wanted to go for native UI. Because why not! It's turned off in Estranged: The Departure now (since it's been released and finished a long time now). This post outlines the mechanics of how it worked - and who knows, maybe I'll bring it back in a future title. šŸ™‚

  1. Markup
  2. Reading in C++
  3. Rendering with Unreal Motion Graphics (UMG)
    1. Downloading Images
  4. Generating Markup
  5. Wrap Up

Markup

The markup itself is kind of like simplified HTML (though represented as JSON for easy reading).

The structure allows for the majority of elements required for a news post - Text blocks, Lists of items, Headings, and Images. There is no inline element support e.g. for bold, italic, links etc. The elements are described as JSON to make it very easy to read on the client in C++. Here's a simplified example:

[
  {
    "title": "Halloween 2020 Update: Waterfront Short",
    "author": "Alan Edwardes",
    "url": "https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/3884878205216116230",
    "published": "2020-10-30T00:01:57+00:00",
    "children": [
      {
        "text": "A new build is live on Steam for Windows, macOS and Linux including a brand new short level!",
        "elementType": "text"
      },
      {
        "url": "https://iamestranged.com/distribute/news/images/68103e6b031a5fb4527317746ea16642.png",
        "width": 658,
        "height": 360,
        "elementType": "image"
      },
      {
        "text": "There is a new short level available in Estranged, called Waterfront. It's a short, story-driven adventure, and is a free and permanent update for the game.",
        "elementType": "text"
      },
      {
        "url": "https://iamestranged.com/distribute/news/images/a8aba228385f16bd1a0d001934ca15ee.png",
        "width": 64,
        "height": 64,
        "elementType": "image"
      }
    ],
    "elementType": "post"
  }
]

To see the full feed for Estranged: The Departure, take a look at latest.json.

Reading in C++

Estranged: The Departure is built with Unreal Engine, which has native libraries for HTTP requests, and simple JSON handling. To read the data back from the server, I used the following logic which invokes an async callback once it gets its result:

/** Header */
UFUNCTION(BlueprintCallable, Category = Actions, meta = (HidePin = "WorldContextObject", DefaultToSelf = "WorldContextObject"))
static void GetNews(const FReceivedNewsDelegate &ReceivedNews, UObject* WorldContextObject);

/** Implementation */
void UInsulamApiClient::GetNews(const FReceivedNewsDelegate &ReceivedNews, UObject* WorldContextObject)
{
	TSharedRef<IHttpRequest> HttpRequest = FHttpModule::Get().CreateRequest();
	HttpRequest->SetVerb("GET");
	HttpRequest->SetURL(BUILD_FRONTEND_ENDPOINT + FString("distribute/news/latest.json"));
	HttpRequest->OnProcessRequestComplete().BindStatic([](FHttpRequestPtr RefHttpRequest, FHttpResponsePtr RefHttpResponse, bool bSucceeded, FReceivedNewsDelegate RefCallback)
	{
		if (!bSucceeded || !RefHttpResponse.IsValid())
		{
			return;
		}

		TArray<FInsulamPostElement> Response;
		if (!FJsonObjectConverter::JsonArrayStringToUStruct(RefHttpResponse->GetContentAsString(), &Response, 0, 0))
		{
			return;
		}

		RefCallback.ExecuteIfBound(Response);
	}, ReceivedNews);
	HttpRequest->ProcessRequest();
}

The Post element is described as follows:

USTRUCT(BlueprintType)
struct INSULAM_API FInsulamPostElement
{
	GENERATED_BODY()

	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString ElementType;
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString Title;
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString Author;
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString Url;
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FDateTime Published;
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	TArray<FInsulamVisualElement> Children;
};

And the Visual Element:

USTRUCT(BlueprintType)
struct INSULAM_API FInsulamVisualElement
{
	GENERATED_BODY()

	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString ElementType;
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString Text; // Text, Heading
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	TArray<FString> Items; // List
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	FString Url; // Image
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	int32 Width; // Image
	UPROPERTY(EditAnywhere, BlueprintReadWrite)
	int32 Height; // Image
};

This can all be interacted with via Blueprints in Unreal, where we can directly read the above data structures.

Rendering with Unreal Motion Graphics (UMG)

In the Blueprint Widget, the Get News call has an event listener which creates a Widget Element for each Post element, then sorts them into two vertical box elements (the two columns of content):

The Post widget populates the Title, Author and URL of the post, and then loops the children, creating the content widget for each:

The "Create Content Widget" function switches on each known Element Type, constructing a widget with the appropriate parameters:

Downloading Images

The Text, Heading and List widgets aren't very interesting, but the Image one is because it must download external content. We also don't want to do so until the widget is actually rendered, to avoid the game downloading all images at once when it doesn't need to do so (a bandwidth problem for the players and myself).

The first thing we need to know is whether the Image element is visible. To do that we can leverage the Tick method, because widgets only tick if they're being rendered. The below logic ensures the image widget has been visible for a total of 100ms, and then calls the Load Image method.

The load image method loads the image in asynchornously, then fades it in once loaded. The implementation just uses the Download Image method available in the engine itself. Usage:

The widget also has a fallback state for cases where the image fails to load.

Generating Markup

The final piece to the puzzle is automation to generate the markup itself. I decided to use the Steam update news as the source, since this is a rich source of data with a simplified number of visual elements (it's limited to BBCode at the time of writing).

This piece is the most complicated piece, because it must:

  1. Obtain & deserialize the Steam News feed from the GetNewsForApp API
  2. Parse the BBCode into a node hierarchy
  3. Traverse the node hierarchy, splitting out block-level elements, joining inline elements
  4. Resize and compress any images found
  5. Write images and the resulting JSON markup to Amazon S3

Here's the main part of the tree traversial logic, all written in Cā™Æ, all running on AWS Lambda:

private async Task<IEnumerable<IElement>> GetElements(SyntaxTreeNode root)
{
    var elements = new List<IElement>();

    var textElement = new TextElement();

    var inlineElements = new[] { "url", "b", "i", "u", "strike", "spoiler", "noparse", "code" };

    foreach (var node in root.SubNodes)
    {
        if (node is TextNode textNode)
        {
            var text = textNode.Text.Replace("\r", string.Empty);

            var previewYoutubeRegex = new Regex(@"\[previewyoutube=(?<videoId>[0-9A-Za-z_-]*);(?<layout>[A-Za-z]*)\]\[/previewyoutube\]");
            var previousYoutubeResult = previewYoutubeRegex.Match(textNode.Text);
            if (previousYoutubeResult.Success)
            {
                text = text.Replace(previousYoutubeResult.Value, string.Empty);
                elements.Add(await ProcessImage($"https://img.youtube.com/vi/{previousYoutubeResult.Groups["videoId"]}/mqdefault.jpg"));
            }

            textElement.Text += text;
        }

        if (node is TagNode tagNode)
        {
            if (inlineElements.Contains(tagNode.Tag.Name))
            {
                textElement.Text += string.Concat(GetTextRecursive(tagNode));
                continue;
            }
            else
            {
                // The tag is a block tag
                if (textElement.Text != null)
                {
                    elements.Add(textElement);
                    textElement = new TextElement();
                }

                if (new[]{"h1", "h2", "h3"}.Contains(tagNode.Tag.Name))
                {
                    elements.Add(new HeadingElement { Text = string.Join(" ", GetTextRecursive(tagNode)) });
                    continue;
                }

                if (tagNode.Tag.Name == "img")
                {
                    elements.Add(await ProcessImage(string.Concat(GetTextRecursive(tagNode))));
                    continue;
                }

                if (tagNode.Tag.Name == "list")
                {
                    elements.Add(new ListElement { Items = tagNode.ToText().Replace("\r", string.Empty).Split(new[] { "\n" }, StringSplitOptions.RemoveEmptyEntries).Select(x => x.Trim()).ToArray() });
                    continue;
                }
            }

            throw new Exception($"Unhandled tag: {tagNode.Tag.Name}");
        }
    }

    elements.Add(textElement);

    return CleanElements(elements);
}

The image resizing/caching logic, using ImageSharp:

private async Task<ImageElement> ProcessImage(string url)
{
    var imageUrl = url.Replace("{STEAM_CLAN_IMAGE}", $"https://steamcdn-a.akamaihd.net/steamcommunity/public/images/clans");

    string hash;
    using (var md5 = MD5.Create())
    {
        hash = string.Concat(md5.ComputeHash(Encoding.UTF8.GetBytes(imageUrl)).Select(x => x.ToString("x2")));
    }

    var bucketName = "estranged-distribute";
    var objectKey = $"distribute/news/images/{hash}.png";
    var finalUrl = $"https://iamestranged.com/{objectKey}";

    try
    {
        var metadata = await _s3Client.GetObjectMetadataAsync(bucketName, objectKey);
        var width = int.Parse(metadata.Metadata["width"]);
        var height = int.Parse(metadata.Metadata["height"]);
        Console.WriteLine($"Skipping image {objectKey} as it is already generated");
        return new ImageElement
        {
            Url = finalUrl,
            Width = width,
            Height = height
        };
    }
    catch (Exception)
    {
        Console.WriteLine($"Re-generating image {objectKey} as it is not valid");
    }

    using (var imageStream = await _httpClient.GetStreamAsync(imageUrl))
    using (var image = Image.Load(imageStream))
    {
        image.Mutate(x => x.Resize(new ResizeOptions
        {
            Mode = ResizeMode.Min,
            Size = new Size(640, 360)
        }));

        using (var ms = new MemoryStream())
        {
            image.SaveAsPng(ms, new PngEncoder
            {
                BitDepth = PngBitDepth.Bit8,
                ColorType = PngColorType.Palette
            });
            ms.Position = 0;

            var putRequest = new PutObjectRequest
            {
                InputStream = ms,
                ContentType = "image/png",
                CannedACL = S3CannedACL.PublicRead,
                Key = objectKey,
                BucketName = bucketName
            };

            putRequest.Metadata.Add("width", image.Width.ToString());
            putRequest.Metadata.Add("height", image.Height.ToString());

            await _s3Client.PutObjectAsync(putRequest);
        }

        Console.WriteLine($"Generated image {objectKey}");
        return new ImageElement
        {
            Url = finalUrl,
            Width = image.Width,
            Height = image.Height
        };
    }
}

Wrap Up

This system was in place for a number of years, serving back news to players of Estranged: The Departure. It was fun to make, and as I stated at the start is something I might bring back if I need this capability again.

The advantage of using (UMG) Unreal Motion Graphics for simple news update UI like this is that it's cross-platform, it could even be deployed to consoles (if network access is available). It breaks down when you need to write something more complex though, and it may be better to embed a browser and drop into HTML.

šŸ·ļø news widget image markup element unreal images elements estranged departure json game logic bring player

ā¬…ļø Previous post: Remembering USB Devices in VMware ESXi

šŸŽ² Random post: Git HTTP Username and Password in Environment Variables

Comments

Please click here to load comments.