Update `tp.web.request` to support non-JSON endpoints

[Yodo9000]

Plugin information

• OS: Windows 10
• Templater version: 2.18.1
• Obsidian version: 1.12.4
• Template folder location: "zz_other/obsidian/templates"
• Trigger Templater on new file creation:
  • Apply the following template:

    <% await tp.web.request("https://jsonplaceholder.typicode.com/") %>

Describe the bug
An empty file is created, and "Templater Error: Error fetching and extracting value" appears as a temporary popup. The console shows the following:

plugin:templater-obsidian:16 SyntaxError: Unexpected token '<', "
<!DOCTYPE "... is not valid JSON
    at JSON.parse (<anonymous>)
    at get json (app.js:1:1066577)
    at Object.eval [as request] (plugin:templater-obsidian:16:676)
    at async eval (eval at <anonymous> (plugin:templater-obsidian:1:1), <anonymous>:5:13)
    at async cn.parse_template (plugin:templater-obsidian:18:75306)
    at async Se (plugin:templater-obsidian:7:258)
    at async cn.write_template_to_file (plugin:templater-obsidian:18:78430)
    at async cn.on_file_creation (plugin:templater-obsidian:22:2510)

eval @ plugin:templater-obsidian:16
await in eval
eval @ VM703:5
eval @ plugin:templater-obsidian:18
Ti @ plugin:templater-obsidian:18
n.wbg.__wbg_call_168da88779e35f61 @ plugin:templater-obsidian:18
$func105 @ 00032406:0xa737
$func27 @ 00032406:0x5c9b
$renderer_render_content @ 00032406:0x8cd8
render_content @ plugin:templater-obsidian:18
parse_commands @ plugin:templater-obsidian:18
parse_template @ plugin:templater-obsidian:18
await in parse_template
read_and_parse_template @ plugin:templater-obsidian:18
await in read_and_parse_template
eval @ plugin:templater-obsidian:18
Se @ plugin:templater-obsidian:7
write_template_to_file @ plugin:templater-obsidian:18
on_file_creation @ plugin:templater-obsidian:22
await in on_file_creation
eval @ plugin:templater-obsidian:22
e.tryTrigger @ app.js:1
e.trigger @ app.js:1
t.trigger @ app.js:1
t.onChange @ app.js:1
e.trigger @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
e.reconcileFileCreation @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
a @ app.js:1
Promise.then
l @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
e.reconcileFileInternal @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
a @ app.js:1
Promise.then
l @ app.js:1
a @ app.js:1
Promise.then
l @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
e.reconcileFile @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
e.reconcileInternalFile @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
a @ app.js:1
Promise.then
l @ app.js:1
a @ app.js:1
Promise.then
l @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
(anonymous) @ app.js:1
n @ app.js:1
Promise.then
e.queue @ app.js:1
e.write @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
a @ app.js:1
Promise.then
l @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
t.create @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
a @ app.js:1
Promise.then
l @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
e.createNewFile @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
e.createNewMarkdownFile @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
e.getDailyNote @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
(anonymous) @ app.js:1
y @ app.js:1
e.onOpenDailyNote @ app.js:1

Expected behavior
The HTML contents of the page are displayed/embedded into the note. The documentation is quite vague to me.

Additional context
I'm not sure if "JSON webpages" is the right terminology for the URL links that do work with tp.web.request.

[Zachatoo]

It's intended use case was for JSON APIs, not HTML.

Note that Obsidian strips out style and script tags, so the imported HTML will almost certainly look strange if we enable this without modifying it in some way, like the Obsidian Clipper does.

Can you help me understand the use case? Do you actually want HTML in your notes or do you want it converted to Markdown? What type of sites are you trying to retrieve HTML from, and why not using a JSON API instead?

[Yodo9000]

I didn't have a specific use-case, I was just wondering what the output would look like for HTML. I can think of a use-case (websites without JSON APIs, dunno how common they are) but that seems a bit roundabout.