Hopscotch project JSON reader

Steps for starters: (a first edition)

Before you start, this involves installing some things so make sure everyone is okay with that. Also there is probably a risk that things might not work, so just be aware.

This is for the program to get a project URL and save its JSON contents in a folder.

  1. You can download the latest Python 3 (this is made with 3.6.0 I think) which will let you have IDLE on your computer. IDLE is what you can use to run Python programs.
    The program I made is using a programming language that is called Python, much like how Hopscotch is a programming language, except it just uses text instead of blocks.

  2. Once you have IDLE installed, open it and make a new file (File>New should do it). Go to the code in the original post, and copy and paste all the code from there into the new file. Then save the file (in a folder where you can find it), with a file name that ends in .py (you could call it hopscotch_read_url.py or something)

  3. In the folder where you saved the file, make a new folder inside there and call it project_data — yep, with underscores and exact case and everything (Python won’t recognise it if you change case or any symbols)

  4. Go back to IDLE and press F5 to run the program that you have open (or go to Run>Run Module at the top menu)

    I am expecting that you might run into an error that will probably say something like ‘No module named lxml’ or something like that. That’s because this program uses some code from a library called lxml (line 4):

from lxml import html

and we will need to install it. Installation was quick for me, but yeah I don’t remember what errors might pop up and what I’ve installed beforehand already :thinking:

5. Find the Terminal shell application for your computer (on Mac, it is the Terminal application; on Windows…tell me if you are using Windows, will have to look it up). And then type in:
pip install lxml
and press Enter. This may take a little while to install.

Then go back to step 4 to run the file again once that’s done.
You will find the project JSON under the project_data folder that you created earlier (hopefully…)

If you run into other errors, please let me know too, I don’t remember everything that might not work :joy: I’m also sorry if things don’t work; maybe I should put this at the start — edit, I will.

7 Likes

We were using GameMaker Language, but it’s not very good for making programs that search for data from currently-obtained local variables stored on your PC/Mac, plus it doesn’t support Apple operating systems very well. I was told to learn and use C++ instead, as it offers solutions to these problems, plus it’s a more common language.

Nope, it’s totally fine! This project seems it would be more fun/successful to develop than other projects I’m currently in the midst of ^^

4 Likes

You know what, I’ll learn Python instead. I haven’t actually touched C++ too much, plus it’ll make embedding your code easier.

I made a GitHub repo just now to store the new project’s code in. Maybe others would be willing to contribute as well :smiley:

https://github.com/TheRealPhotonic/Hopscotch-JSON-IDE

4 Likes

Cool, thanks for the tutorial! :)

1 Like

Is this like the Hopscotch JSON editor app you were working on a while back?

2 Likes

Yep, it’s that project, but I’d like to rework it with Python. I’ve found that GML isn’t very good with these types of things :)

3 Likes

Yeah I’d initially learnt Swift as a programming language but couldn’t see much about using it for more general things (everything I’d done was in the context of apps). Then got involved with Python (after going to a computer science summer school) It is very readable and quick to get your ideas out :slight_smile: (not as fast for me as rapidly dragging out blocks on Hopscotch but what you write describes what is actually directly going on).

also learnt some more about programming languages in general — and I think that learning any is great, really :slight_smile: And also I was able to get a local version of Discourse on computer:open_mouth: so I was thinking, just maybe, we could do something for the forum…

Discourse uses Rails and Ember.js I think… (I am not very sure at all) but Discourse is open source…and it would be interesting to see what we could try :slight_smile: I only made the initial thing in Python because i’ve been a little more familiar with its capabilities (as a general purpose programming language), but was actually intending to see if JavaScript could do similar things after that, or I’ve been peeking around Discourse for a bit too (including this topic How to start building stuff for Discourse if you're newbie (like myself) - tips & tricks - Discourse Meta)

With embedding something as a plugin for the forum (again, this is a total dream :stuck_out_tongue:), I was thinking more of just viewing code at first (editing projects and project JSON might be an extra step from there) but boy, an IDE would be awesome :slight_smile: With web apps, they’re accessible for more devices (as opposed to an app just for iOS/Android)

When I was at the computer science summer school (ill probably end up mentioning it a lot, because it’s thanks to it that ive been able to do all this really), we made a social networking website that had a server using Python, and some front end stuff done with HTML, CSS and JavaScript, plus ive been familiar with working in some parts of it, so I can see a path towards making a web app sort of :slight_smile:

I think at this point, I might just aim for being able to display code from projects as a first starting point hehe :slight_smile:

so some steps are:

These have now been moved to this post:

Hopscotch project JSON reader - #30 by t1_hopscotch

but here is what was here:

  • familiarise myself with some more of the structure of the project JSONs, about plugins in Discourse, and some things related to how I think displaying the code might work (including template engines, can talk more about this later)

As for the actual viewing, general idea at the moment is:

  • get list of rules in the JSON file (can be accessed by data['rules'])
  • the blocks for rule can be accessed by:
ability_id_for_rule = rule['abilityID']

FOR each ability in data['abilities'] DO
    IF ability['abilityID'] equals ability_id_for_rule THEN
        blocks = ability['blocks']
    END IF
END DO

(I might ask @BuildASnowman to check that bit^^ :slight_smile: for the structure of the Hopscotch JSON files) This isn’t in any particular programming language (it is pseudocode, mostly meant for any language in general so you can understand if you know any)

  • then we could have a whole list (maybe) for all the blocks and rules in the project, with each rule in this order:
name_of_when_block = rule['parameters'][0]['datum']['description'] // I will have to check this for other Whens
blocks = (code from above)

and we could have a whole list for the rules, then another whole list for all the objects.

Just realised how big this can turn out :slight_smile: Not sure about a mechanism for separating all these (e.g. in Hopscotch, the rules for each object are separated) but at this stage, just more interested in getting to something that might work :stuck_out_tongue:

  • then from there, there is the rendering of how the blocks will look.
FOR each block in blocks DO
    block_display_text = ''
    block_name = block['description']
    block_display_text += block_name

    FOR each parameter in block['parameters'] DO
        parameter_name = parameter['description']
        parameter_value = parameter['value']

        //TODO: Colour each accordingly to what they are (e.g. parameter_value will have a white bubble around it in Hopscotch) Would probably use template engine here
        // This is where you can use CSS classes and things :)

        block_display_text += ' ' + parameter_name + ' ' + parameter_value
    END DO

END DO

(okay im being a bit naughty here and using some conventions specific to Python)

The template engine will be here for generating HTML from this (if anyone is interested in styling/graphics or HTML & CSS, come here!!) Then from there, there will just be displaying the HTML.

Okay, wow that was pretty satisfying listing a general line of ideas.


Oh, I meant to say also, from working on that project at the computer science summer school, I also saw that you can learn as you go and search up stuff as you need it, so if you have the general concepts from trying to learn programming languages (Hopscotch already gives you a lot :))

I don’t think you’ll have to go and learn a specific language (e.g. Python) more deeply, but I can see that you think it would be a good idea to do so :relaxed: so I won’t impose advice on you or anything in the least :stuck_out_tongue: (other people have told me ‘once you learn one, you learn them all’ but people’s own experiences mean a lot than just general words of advice, as ive found :))

And wow I think it’s really awesome you want to work on this too!!! Hehe, might not be successful, but we’ll probably learn a lot along the way :blush: and yes, have lots of fun!

I am just sharing the general ideas I had basically, always ideas are very welcome, and it’s lovely being able to work on something together with other people (also just come and go any time :slight_smile:)

5 Likes

More information on lists and dictionaries (for JSON file), and template engine

I’ll put more explanation of some data structures and things here. This is more just to give a general idea of what you can do with these:

(to be updated)

Lists

my_fruit_list = ['apple', 'orange', 'banana']
>>> my_fruit_list[0]
= 'apple'
>>> my_fruit_list[1]
= 'orange'
>>> my_fruit_list[2]
= 'banana'

Dictionaries

my_favourites_dictionary = {
'colour': 'blue',
'movie': 'Frozen',
'app': 'Hopscotch',
'hobby': 'coding'
}

You can access items by just specifying its key:

>>> my_favourites_dictionary['colour']
= 'blue'
>>> my_favourites_dictionary['app']
= 'Hopscotch'

TODO: :hammer_pick: explain lists and dicts in context of the Hopscotch JSON files.


Template engine

— something used for putting code for HTML basically

it allows you to use loops, variables, and if statements and things with HTML. I shall put some examples of things you can do.

These following conventions are specific to particular engines:

// Say you have the list of fruit from earlier on, and you want to display each fruit in a HTML paragraph

<p>This is my shopping list for the fruit salad I'm making.</p>
{% for fruit in my_fruit_list %}
    <p> Item: {{ fruit }} </p>
{% end for %}

^the template engine will turn this into:

<p>This is my shopping list for the fruit salad I'm making.</p>
<p> Item: apple </p>
<p> Item: orange </p>
<p> Item: banana </p>

You could also do things like this:

{% for fruit in my_fruit_list %}

    {% if fruit == 'apple' %}
        <p style="color:red;"> Item: {{ fruit }} </p>
    {% else if fruit == 'orange' %}
        <p style="color:orange;"> Item: {{ fruit }} </p>
    {% else if fruit == 'banana' %}
        <p style="color:yellow;"> Item: {{ fruit }} </p>
    {% end if %}

{% end for %}

don’t worry if the syntax & punctuation looks confusing, it does what it says it does basically :slight_smile: (if the fruit equals ‘apple’, then colour it red. if the fruit equals ‘orange’, then colour it orange. and if the fruit equals ‘banana’, then colour it yellow.)

^this will become:

<p style="color:red;"> Item: apple </p>
<p style="color:orange;"> Item: orange </p>
<p style="color:yellow;"> Item: banana </p>

(maybe I could have even used <ul> with <li> as for the examples :joy: but more people will know what <p> is, I guess :slight_smile:)

4 Likes

Outline for displaying JSON files

This is just extracted from earlier.

###Quick general idea:
If I could make this, how would it work? sort of thing:

  • if you type in [hopscotch view] http.c.gethopscotch/projecturl [/hopscotch] then some HTML will be inserted into the page for the blocks that are in the project.

(this is for if finding websites (would be yes, look at onebox) is possible and if extracting <div> from their HTML is possible using the language that the forum does. Note to self, can look through discourse-spoiler-plugin for detection of [thing][/thing] using regex)

Breaking down into more steps…

  • Search post for any [hopscotch view] or [/hopscotch] and remove when render post. Extract the URL from the middle of the tags, ignoring whitespace. (This sort of thing can be done using something called regex — regular expressions.)

  • (will grab general steps from earlier)

get the html from a project url
look for <div> tag that has a `data` attribute
get the value of that `data` attribute (as a string)
convert to an object (or anything that you can manipulate)
// save the object to a JSON file

Viewing of code:

As for the actual viewing, general idea at the moment is:

  • get list of rules in the JSON data object (can be accessed by data['rules'])
  • the blocks for rule can be accessed by:
ability_id_for_rule = rule['abilityID']

FOR each ability in data['abilities'] DO
    IF ability['abilityID'] equals ability_id_for_rule THEN
        blocks = ability['blocks']
    END IF
END DO

This isn’t in any particular programming language (it is pseudocode, mostly meant for any language in general so you can understand if you know any)

  • then we could have a whole list (maybe) for all the blocks and rules in the project, with each rule in this order:
name_of_when_block = rule['parameters'][0]['datum']['description'] // I will have to check this for other Whens
blocks = (code from above)

and we could have a whole list for the rules, then another whole list for all the objects.

Just realised how big this can turn out :slight_smile: Not sure about a mechanism for separating all these (e.g. in Hopscotch, the rules for each object are separated) but at this stage, just more interested in getting to something that might work :stuck_out_tongue:

  • then from there, there is the rendering of how the blocks will look.

TODO :hammer:

  • do this as template language format underneath. the JSON data object will be passed in as context variable
FOR each block in blocks DO
    block_display_text = ''
    block_name = block['description']
    block_display_text += block_name

    FOR each parameter in block['parameters'] DO
        parameter_name = parameter['description']
        parameter_value = parameter['value']

        //TODO: Colour each accordingly to what they are (e.g. parameter_value will have a white bubble around it in Hopscotch) Would probably use template engine here
        // This is where you can use CSS classes and things :)

        block_display_text += ' ' + parameter_name + ' ' + parameter_value
    END DO

END DO

(okay im being a bit naughty here and using some conventions specific to Python)

The template engine will be here for generating HTML from this (if anyone is interested in styling/graphics or HTML & CSS, come here!!) Then from there, there will just be displaying the HTML.


Ideas related to editing:

I am not this far yet :joy: but just had some ideas for maybe related to what @Rawrbear is interested in trying :relaxed::blush:

  • Hmm I am not too familiar with what the JSON will contain when you start from scratch for example (and if/how it differs for a project that is published vs a project that is unpublished — you can view the JSONs for both, for unpublished ones, it involves connecting the device to iTunes I think)

  • publishing will be different to editing, so it looks like it will just be editing for now

  • Also BuildASnowman mentioned earlier :smiley::

4 Likes

Updated one of the previous posts :slight_smile:

5 Likes

I’ve strayed away from coding with text.
It seems like you will eventually make your own coding app by the skills you are showing.

3 Likes

Hi Stradyvarious :blush:
haha it’s actually just because I got to learn so much while I was at the computer science summer school (I don’t really know about level of work or whatever —personally I try not to classify things based on ‘difficulty’, but I just know it was lots of fun and learnt lots) and I think im just excited to get to use some of things we looked at.

Yeah initially for a lot of time, I was really finding coding with text more cumbersome — I guess then after trying to learn Swift for a bit of time each week over a year, then two little programming competitions and a work experience with Python, some basic stuff with Xojo & Visual Basic at school, playing around in Lua on Codea, and finally this project that we did while we were there, all were text-based programming so I think ive just fallen into it.

We are actually using a drag and drop language based on Scratch for one of our classes (called Edgy) and it has graphs/networks (as in people and friendship type networks), it is great for that; compared to if we did in in python for example, where we’d have to get graphing libraries that makes it a little cumbersome. (Oh, it also has dictionaries, stacks and queues as well)

I think I just like whatever is fast for thoughts and ‘clean’/easy to read; I was thinking a bit over programming languages during the summer school after seeing some people answer when asked what their favourite was. And at the time I was thinking, I don’t really have a favourite or anything — I like the readability of Python, I like how Lua is ‘lightweight’ and relatively fast to make stuff in, I like the things you can do with JavaScript, I like how ‘safe’ Swift can be (with explicit declaration for types and optionals possible) and of course I love pulling out blocks in Hopscotch at the speed of thought (I admit maybe this was strongest for the last update before the iPhone version was made universal, but was still fast like that). I admit ive kind of leaned more into Python the more time I spend on it since then :joy: but I think that’s just more natural bias (it would probably be that way the more time I spent with most things. except it kind of has meant with Visual Basic, the time to get something started is a little agitating)

If there were more capabilities that could be done using drag and drop, I might actually just use it generally; at the moment im just using things based on whatever is needed (was switching between Python and JavaScript while working on our group project). I also love the accessibility with drag and drop — making coding accessible for heaps more people. And also not having to bother about extraneous syntax errors (‘you forgot a colon here!’) and having blocks in itself is very, very readable/understandable — it is pretty nicely close to natural language and describes what it does pretty much. And you don’t have to type out a whole thing each time (e.g. set angle to ___), you just have a block that has all the text for you.

wow looks like I was thinking over quite a few things :joy: But yeah :relaxed: And I still see what you’re working on, and I really think that everything you’re doing and exploring is awesome — really :blush:

3 Likes

I see why it’s called “Snapapps”
That is a tweaked version of Snap.
What I like about Construct 2 is “Behaviours” that can easily be added to a Sprite.
Like "Solid,platform character,line of sight,turret, among others.

You probably know how much code you have to use in JavaScript just to get a Sprite to always point towards the cursor.

And trying to code a Sprite to act like a solid object.(massive headache)

2 Likes

Yep Edgy is based on Snap :smiley: (which is based on Scratch I believe?) the class we’re using it for is called Algorithmics, basically a sub-line describes it: ‘the thinking before you code’

(thinking about doing these manually — yep, would be bothersome :persevere: I haven’t actually used JavaScript that much, just very briefly for some general web site functionalities e.g. forms)

Also while learning all these text-based programming languages, it made me see how Hopscotch made all these concepts really accessible — for example, if you’re using a For loop in a text based language with brackets or starting and ending words, not only are you first introduced to the idea of a loop (something that repeats code) but you also have to note the syntax (‘put a Do here and a Next down the bottom’) and recognise what the errors are when you don’t do these; it’s more things if you’re learning for the first time.

3 Likes

I wonder if you could access/edit a JSON folder for Tynker?
What you are doing with Hopscotch might be possible with Tynker.

3 Likes

Oooh that is really interesting :open_mouth: for Hopscotch, I know that the project JSONs can be found by plugging the device into a Mac then opening iTunes and viewing the files; and I know some other apps have data that can be accessed this way too

4 Likes

It’s possible for me to write something up in C++ if I learn more of it, but if anyone else wanted to contribute to the IDE (such as you or anyone else who felt like adding onto the project), Python is a widely-known language that you can make “mobile programs” with; they can be both embedded into web applications and installed onto computers, can’t they? I find that pretty neat :)

Yep, I’m pretty sure sam mentioned in the Rails development topic you linked that it’s very difficult to make a Rails app - from installing a Virtual Machine to develop to having long loading times (for something? It wasn’t clarified very well when I skimmed over sam’s post), it’s a difficult process to make something from Rails. We could still nonetheless try to cook something up, if a Discourse plugin is what we’d like to make as well :)

What we (by we, I mean @K1ish and I, the developers of the original IDE project) planned to do for the first GML IDE project we wanted to make was to:

  • Gain an API from Hopscotch to log into a Hopscotch account, then fetch .json code
  • After we receive the .json code, find a way to convert that to visual block code

Coincidentally, @K1ish and I somehow found out that we went to the same school district :o, so we were then oficially allowed to talk behind the scenes in email by the Community Guidelines’ terms. Unfortunately (and/or fortunately), when he emailed Liza about it, he received this response:

(The “Here’s a pic” picture redirects to a picture of the iTunes screen you’d see when navigating to the Hopscotch Documents directory.)

I know @BuildASnowman has done something very similar to this with his MIDI music hack; maybe he’d be willing to help out with something in general with finding the algorithm of the blocks and how they’re laid out in the .json files. :)

Yup, exactly, Hopscotch has given me a ton of insight in terms of programming. Searching up your particular problem on Google offers great solutions to problems you encounter :smiley:

Well, I’ve been planning to learn a bunch of programming languages before college anyway (including C++, C#, JavaScript, jQuery, HTML/CSS in-depth, Python, SQL, etc.). Python is definitely a great language to start it all with :D

Yeah, I’m definitely having fun learning the language and planning the development of this project here on this topic ^^ Let’s see how far we go before we crash and burn, if we do. xD

Yep, the ideas you’re sharing out are super useful! Thanks a lot for contributing to that factor as well :)

2 Likes

Oop, didn’t see this post - thanks a lot, @BuildASnowman! You’ll definitely be a big help to this project at the pace we’re moving through all of this at :)

2 Likes

yeah, I am not sure about making Python programs that can be installed, but definitely with something that can be used on the web — using Python as a server :relaxed:

hehe yeah Discourse is a Rails app; whoops I think I should have mentioned that link was just for getting a local version of Discourse for testing and not adding anything (and yes it is quite a few steps, I should have linked the blog post rather than the discussion topic anyway though :sweat: I don’t understand pretty much anything of what they’re troubleshooting either — all good :thumbsup: :slight_smile: :joy: sorry about that being confusing :sweat_smile:). this one just makes it more approachable :slight_smile::

(the first part of the blog post has a pic that describes how this may feel pretty well actually :joy:)

pic (just for showing)


anyway, that is just how I was able to run a local Discourse on computer, not needed right now :slight_smile: but definitely there if anyone is interested!

I just have an example for what people can do with plugins for Discourse, like here is part of the code for the [spoiler] feature here on the forum (it is in JavaScript)
https://github.com/discourse/discourse-spoiler-alert/blob/master/assets/javascripts/spoiler.js

I am just looking up a few things in JavaScript at the moment :slight_smile:

but yeah I think Ill keep looking around for a bit longer :relaxed: and again these are just dreams hehe :stuck_out_tongue_closed_eyes:

And wow — with your discussions with K1ish and THT — that is really interesting! I see, I think it’s a good idea asking about an API, and yep I have seen the steps to view from iTunes too :relaxed: (though have not done it myself yet)

Yep (me: looks around for everything on stack overflow :joy:)

you are feeling really motivated and want to learn lots, I can see!

:slight_smile::blush: I see you’re enjoying this so far too :relaxed: I just want to say, I think it’s lovely seeing all this — you wanting to participate where you can. Plus would like to stress two things: it is very okay not knowing anything (I’m definitely going into this blind) and there is no such thing as a silly question or statement: so ask away whenever :slight_smile:


edit: should mention, aside from Discourse plugin, there is also making a website with python as server — that I can see more of a path to at the moment :slight_smile:

edit 2: yeah I think can probably end up going with this, or checking what GitHub pages could do too

  • basically less things to worry about with other aspects of Discourse – having more things to look over, plus probably some limitations (where you can add stuff)
  • also if we do get something working at the basic, will make it a smaller step with translating/integrating it

I want to make a whole bunch of topics to keep track of things (I just somehow like the space…) :joy: it’s going to feel overboard but since we’re keeping communications to forum, it will just be like that :slight_smile:

Let’s see…some of the ideas that I had:

  • timeline topic of project where can update what I’m doing/we’re doing as we go (when we commit something, we can reply there with link)
  • todo list in general for different subsections
  • outline of how things will work, plus break down of steps
some things I need to put for outline (really quick notes so these will make no sense right now)

Python tornado server, server.register(post), have <input> for a project URL and get the value inside and set that as the URL, then try to display/print JSON to check. Then look through Django template engine.

  • asking for help on different sections of project - e.g. when I call on people for design
  • documentation and explanations, how to run things and so on, and things related to structure of JSON file
6 Likes

Yes, you can edit JavaScript so that it says something else. If you are on the forum, you can edit someone’s post.

4 Likes