# Extracting forms from a web page

**URL:** <https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224>\
**Category:** Show & Tell\
**Tags:** markup, html\
**Created:** [July 25, 2017, 4:08am UTC](https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224 "2017-07-25T04:08:27Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![rgchris](https://revolt.metaeducation.com/forum/user_avatar/revolt.metaeducation.com/rgchris/32/14_2.png) [@rgchris](https://revolt.metaeducation.com/forum/u/rgchris)\
**Post date:** [July 25, 2017, 4:08am UTC](https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224/1 "2017-07-25T04:08:27Z")

</div>

Having just reached the first stage of developing a [module for parsing HTML](https://github.com/rgchris/Scripts/blob/master/experimental/markup.reb), I thought I'd share a wee snippet for gathering forms from a page to illustrate how the module can be used. The following function:

- Loops through the page content copying everything stored between `<form>...</form>` tags
- Loops through each form picking out the pertinent information from `<input>` and `<textarea>` tags

Will leave as an exercise as to improve on this:

```rebol
import <markup>

gather-forms: func [
	doc [string!]
	/local forms mark extent attributes
][
	doc: load-markup doc
	forms: collect [
		parse doc [
			any [
				<form> mark: some [
					and <form> (print "Warning: cannot gather nested forms") break
					|
					extent: </form> (keep/only copy/part mark extent) break
					|
					end (keep/only copy/part mark tail mark)
					|
					skip
				]
				|
				skip
			]
		]
	]

	collect [
		foreach form forms [
			either map? pick new-line/all form true 1 [
				keep make object! [
					action: select form/1 "action"
					method: any [select form/1 "method" "GET"]
					fields: make map! collect [
						parse form [
							any [
								<input> set attributes map! (
									keep lock any [select attributes "name" "anon"]
									keep select attributes "value"
								)
								|
								<textarea> set attributes map! set text string! (
									keep lock any [select attributes "name" "anon"]
									keep text
								)
								|
								skip
							]
						]
					]
				]
			][
				print "Warning: form has no attributes"
			]
		]
	]
]

probe gather-forms to string! read https://rebol.metaeducation.com

```

---

<div class="post-metadata">

**Author:** ![gchiu](https://revolt.metaeducation.com/forum/user_avatar/revolt.metaeducation.com/gchiu/32/22_2.png) [@gchiu](https://revolt.metaeducation.com/forum/u/gchiu)\
**Post date:** [July 25, 2017, 4:22am UTC](https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224/2 "2017-07-25T04:22:47Z")

</div>

I had a r3/view [script](https://github.com/gchiu/Rebol3/blob/master/scripts/loginso.reb) which would parse out all the forms on a page and allow you to enter values to see if you could post to it.

---

<div class="post-metadata">

**Author:** ![gchiu](https://revolt.metaeducation.com/forum/user_avatar/revolt.metaeducation.com/gchiu/32/22_2.png) [@gchiu](https://revolt.metaeducation.com/forum/u/gchiu)\
**Post date:** [July 25, 2017, 4:39am UTC](https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224/3 "2017-07-25T04:39:20Z")

</div>

Any reason why you use `func` instead of the auto-collecting locals `function` ?

---

<div class="post-metadata">

**Author:** ![rgchris](https://revolt.metaeducation.com/forum/user_avatar/revolt.metaeducation.com/rgchris/32/14_2.png) [@rgchris](https://revolt.metaeducation.com/forum/u/rgchris)\
**Post date:** [July 25, 2017, 4:40am UTC](https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224/4 "2017-07-25T04:40:36Z")

</div>

Well—Red has a GUI, wouldn't be a leap to make that work!

However, even just in a script the above is close to that:

```rebol
the-form: first gather-forms some-web-page
... change some form fields ...
write to url! the-form/action [
    the-form/method
    to-webform to block! the-form/fields
]

```

_Again, left as an exercise to resolve relative 'action urls._

_Note to self: update_ `<webform>` _to allow_ `map!` _arguments._

---

<div class="post-metadata">

**Author:** ![gchiu](https://revolt.metaeducation.com/forum/user_avatar/revolt.metaeducation.com/gchiu/32/22_2.png) [@gchiu](https://revolt.metaeducation.com/forum/u/gchiu)\
**Post date:** [July 25, 2017, 4:45am UTC](https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224/5 "2017-07-25T04:45:14Z")

</div>

how about including the `gather-forms` function into the module?

---

<div class="post-metadata">

**Author:** ![rgchris](https://revolt.metaeducation.com/forum/user_avatar/revolt.metaeducation.com/rgchris/32/14_2.png) [@rgchris](https://revolt.metaeducation.com/forum/u/rgchris)\
**Post date:** [July 25, 2017, 4:46am UTC](https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224/6 "2017-07-25T04:46:13Z")

</div>

1. Habit.
2. Get dinged when I use words I hadn't accounted for.

---

<div class="post-metadata">

**Author:** ![rgchris](https://revolt.metaeducation.com/forum/user_avatar/revolt.metaeducation.com/rgchris/32/14_2.png) [@rgchris](https://revolt.metaeducation.com/forum/u/rgchris)\
**Post date:** [July 25, 2017, 4:50am UTC](https://revolt.metaeducation.com/forum/t/extracting-forms-from-a-web-page/224/7 "2017-07-25T04:50:23Z")

</div>

'Cause it's a quickie and really could use some work—just thought I'd share it as it's common request and I figure it's a good start point if anyone wants to run with it now.

I've something better in mind anyhow.

BTW: should have a Cookbook/Reviews category in the forum.
