Proposal for the future of QCubed - to the QCubed Community
All,
I am concerned that QCubed is in trouble. QCubed originally existed because so many developers really loved QCodo, but either were frustrated with the lack of progress on it, or wanted to see it go in a new direction. Well, here we are again in a situation where there is little progress. I have contacted some of the developers in the core contributors list. Some have said they have moved on to other things, and some never responded. A couple are still lurking on the forums, which is GREAT. The ones that responded to me suggested I take the lead to make the next step, so this is my attempt.
I have been working on changes to the framework which I think would make a good 2.1 release, and which is important to future work I am planning. I would like to check it in to SVN in the experimental section to show you all and get help with it, but there are server problems that are preventing that. I believe some others have pointed out other server problems, but the server owner is unresponsive to my emails and the other posts on this forum.
So, to prepare for a QCubed 2.1, we need to get our house in order a bit, and those of us who are still interested in moving QCubed ahead need to help. It will take some work, but in the end, I think we will have a great platform to build some great apps with.
Here is my proposal. Note that this is a straw man to promote discussion. If you have a vested interest in the future of QCubed, chime in.
- Move the Web site to a public code hosting platform, so that it never has to rely on one person to maintain the basic development process. I don't have a lot of experience with these, so some of you who have worked with these platforms, please help. Before we decide on a platform, we need to decide what we want. Please see below for my thoughts on platform goals.
- Take QCubed further in its integration with JQuery UI. I have been working for a couple of weeks on this. There is much to do here. The primary end-goal here would be to have the code generation process by default generate forms that have JQuery UI objects in them, and have these objects be generally accessible and responsive through PHP code.
- Finally, finally, finally get our documentation in order. I have thoughts on this too. Please see below.
Web Hosting Goals
The success of any open source project is dependent on a group of people working together and over time, building trust in each other. The quality of the people and the quality of the trust create great software. I would like to see QCubed eventually become a well recognized platform with many developers actively moving it forward. That will require an environment that encourages creative thinking and experimentation, open discussion, and relationship building. There must be few barriers to posting ideas, code and documentation, and it should lean more on allowing people to make mistakes, and making it easy to back those out, rather than requiring lots of hurdles to jump before you can get code into the core.
I am not sure there is a particular public solution that would do this. We might have to build it. But before then, we should see if a public site would work. SourceForge comes to mind. GitHub maybe, but its not a complete solution, and the complexity of GIT might be a barrier. KForge could work, but it needs to be hosted by someone, and that host would need to be comfortable with having multiple people have root access so that we don't get into the problem again that we are in now.
Does anyone have any thoughts on this?
2.1 Features
Before discussing 2.1, I think a look at the present and look far ahead is in order. There are many PHP frameworks out there. Many of us who use QCubed, and used QCodo, know that this framework excels as a RAID development tool. There is nothing like being able create a complicated set of SQL tables, and pressing a button and seeing an entire application get spit out that is already functional, and just waiting for us to tweak it. QCodo was originally designed to get you far along the process of building a site, and then you took it from there.
But of course, things change. Once you build a site, you might add fields, or whole tables. The underlying framework may get improved. PHP is evolving, HTML is evolving, Javascript is evolving, browsers are evolving, and whole new platforms (e.g. iPhone, iPad) are being created. Recent changes in the architecture of QCubed and QCodo have made it easier to rely on the framework as an ongoing platform to host a Web application. The idea is that your application should be able to easily incorporate changes in the framework and changes to your data model so that the ideas and the data encapsulated in your application can live on as the world changes.
The recent effort to incorporate JQuery UI points QCubed in a direction of relying more and more on the work of other frameworks and tools, to further incorporate all the work of the thousands of developers working on complementary projects. This is a really important shift, in that it helps us to focus on what QCubed is good at, without trying to do everything ourselves.
So, to encapsulate the idea of QCubed's niche in the world, I would say that it is a fantastic tool for quickly building data driven Web sites where high performance is not a major requirement. This makes it a great tool for building in-house intranet sites, for small-businesses who have modest traffic but complicated data requirements, and for rapid prototyping. QCubed is a great tool on the belt of a software consultant. I see nothing wrong with sticking to these goals and with that audience. QCubed has a lot of room to grow with just these goals.
The next steps for QCubed I see are 1) to further integrate jQuery UI, 2) to incorporate the changes at QCodo that make it easier to upgrade to future versions of the framework without it being a big pain, and 3) to incorporate some of the recent code people have suggested to make it easier to get results from javascript requests. There are lots of other improvements that could be incorporated as well.
I am building a layer that sits between the generated JQuery UI code and the user code so that the JQuery UI objects are easily accessible from QCubed code. This opens up a LOT of functionality that users on these forums have requested, including:
- Being able to code generate forms that are jQuery UI based.
- Making forms Themeable.
- Using Autocomplete to manage selection from large lists easily.
- Much improved drag, drop and resize functionality.
- An easier process to create widgets for specific situations that are based on JQuery UI objects.
- The easy addition of jQuery UI widgets that are in planning over at the jQuery UI site.
What are your thoughts on what QCubed is good at, and the next steps?
The Documentation Problem
Most people hate writing documentation. I think the problem is primarily that after you put so much time into writing code, you realize that you may have to put in an equal amount of time to document it, and that is just not as fun.
The code hints in the comments help, but they are not a replacement. The examples site is great, but its not a replacement either.
What I think we need is a kind of living documentation system that allows all of us to contribute to the documentation on an ongoing basis. As you learn about some part of the framework, you may discover something that you want other people to know about (or that you want to remember and need a place to write it down.) I really like the PHP site's documentation, which allows people to leave comments on the doc. But you still can't easily change the actual documentation part. What we need is a kind of wiki that lets us write the documentation as we go, but also leave comments on the documentation as well. I would like it to be editable by anyone with a login, and not just a few select people. Because of the potential for abuse, it needs to be regularly backed up and have some form of version control, but my experience is that such abuse is rare and we should deal with it when we have that problem.
What are your thoughts on the documentation problem? Are there any collaborative documentation products out there that could help? Is there a public site that hosts something like that?
SP, I'm not exactly sure just how I can help/participate at this time. Frankly, I'm a little surprised your post has been up for two days and no response yet. I imagine the core developers, who I know have been working very diligently for the past two years that I've been around QCubed, are likely getting a little burned out and need to convert some of what they've done into income.
Anyway, at this point, I have a vested interest in seeing, helping, promoting the advancement of QCubed. Although I've had my problems with it, QCubed has certainly made life much easier for me. I evaluated several other frameworks and QCubed looks even better to me today than it did two years ago. I'm still working with 2.0.1 (yea I know, I'm probably the only guy working with that version), and would like to start moving over to 2.0.2 soon, but I've been a little afraid to move, until I know it's really stable and there aren't some major changes in architecture just waiting to be applied in a month or so. I sort of made that mistake, moving from 1.1.1 to 2.0.1.
I will be very happy to help in any way I can. From your post, it sounds to me like you have a good grip on a good direction. If you want to contact me directly, please feel free.
Thanks for starting this thread.
As you noted, the current core contributors group has been a bit burned out - folks are really busy with their commitments. To me, the most important thing for the success of the framework is the PEOPLE that are willing to invest time. If that piece is there, everything else is solvable.
The key is establishing a new, small, fully committed group that's constantly contributing to the framework and has a final say on decisions. Let's call this group "core contributors." Minimum requirement for core contributors will be to invest no less than 20 hours per month on the framework (that's an average of 5 hours a week).
Being a core contributor is a rotating role; it's not an ongoing commitment. You can nominate yourself to be a contributor for a minimum term of 1 month. You can stay active for as long as you have the time - but you must active during the time of your term.
I'm calling all current active users of the framework. QCUBED NEEDS YOU! Please step up to the plate and volunteer to be a core contributor for a couple months. Be a part of the group that will decide the future of this framework and make the next few releases happen!
As of right now, we have two committed core contributors:
- Kristof for the months of July and August
- Me (alex) for the months of July and August
Will you join us? We need your help!
Yeah, I wish I had time to spend working on QCubed more often, and we certainly still rely on the framework for our business, so you won't see me go away entirely, but QCubed is mostly at a point that I'm happy with it, so my focus has been on our application rather than the framework. :)
As for site hosting, I will say that I personally like the infrastructure we have right now. I find trac is perfect integration between svn and tickets, the api site has official documentation generated from the code comments (which anyone can change via tickets) and it supports comments from anyone that has registered for the site. I'm not clear on any benefits a third party would provide us.
We definitely would like someone to step up and fix any server issues we do encounter, but at the moment, I think everything is functioning correctly.
The only other comment I would like to make is that I do see an opportunity for QCubed to lean down some, and become a framework better suited to performance-centric sites as well. There are some open tickets discussing some of these possibilities already.
On the server issues, there have been posts in the forum regarding some problems. What I notice are:
1) I can't check in to the experimental area. I get a MKACTIVITY error, and looking online, it likely indicates that the server permissions are not set correctly.
2) The auto-generated online documentation on this Web site is not working. All the links are broken.
Who is setting milestones, assigning bugs, checking in the bug fixes posted on this site, tagging the source for release, building and posting the releases, etc? If some kind of special permission is required to do these core contributor tasks, and the core contributors go away, we are all sunk. Even worse if only the server owner does this.
Trac and SVN are fine, but if they are all on a private server, the entire project depends on the server owner. What if the owner has an accident, what happens to the project? A public server with multiple people having master access will protect all of our investments in the time we put in.
I am very willing to be a core contributor. Having this role be a rotation is great. But again, we need to think about what is the right way to enable this kind of development effort.
You are right that people are the key, but they don't come out of no where. To attract new developers, we need to really work on the experience they have when they first come to this site. We need to do a little promotion here. The home page needs some work. The tutorial is fantastic. The doc needs some work. But the current process of fixing bugs and proposing new features feel like its getting in the way of attracting new developers. But perhaps, I don't really understand the process.
I may get burned for this suggestion, but why not put it on GitHub? Then it is so easy for ANYONE to fork the project and then issue a pull request if they thing they have added extra functionality or taken care of a ticket. Then the core contributors accept or reject the pull requests and everyone is happy. The ability for anyone easily to fork and branch, and contribute has helped all sorts of projects take off.
Edit: Apparently I didn't finish reading the original post. So just know that I vote for GitHub. I'd also be happy to be a core contributor in August and September. Put my name on the list.
Folks,
I'd like to stress yet again that people are everything. People that are the core contributors for the project will decide EVERYTHING about the project, including where it is hosted, how the documentation is done, how the forum is moderated. Everything. Core contributors will have the full power to make all the calls on the destiny of the project. If we decide on Github - it will be github. If we decide that the current server sucks - we will change it.
The core contributors will have full control over everything. The rest of the folks in the community will have an advisory voice - and the core will listen, but will make the final calls.
Edit: I am SO thrilled to hear that you guys are willing to step up for the core contributor role. I'd like to invite a couple more folks to help us out in July/August. Anyone up for it?
Hello,
I would like to contribute but unfortunately won't have much time in July. At least not 5 hours per week, but should be OK in August and definitely after that in October/November.
Cheers
Helge
Excellent thread and it's great to see this engagement!
Here is what I think:
Moving the code to GitHub would be awesome indeed. That would make it easier for non-core developers to work on code and create patches and then request pulls from the core devs. Not to mention the additional possibilities git offers (although that's not really an issue with git-svn).
I want to contribute to this project and give back something. Even though my skills probably aren't high enough for core contributing, I am sure that I can assist with whatever there is to do that I am able to do.
It would be nice to see more people on IRC (#qcubed on freenode or "Chat" in the page navigation above). I am usually there and sometimes help people if I can (and more often than that I get help from Vexedpanda or alexxed. Thank you very much, you rock!). I would love to see more people joining in the future and maybe it can even become a place where we can coordinate things related to the advancement of the QCubed project in realtime :)
As I mentioned above, I'm very committed to using QCubed for the long term and want to contribute, where I'm competent to do so. I'm just starting to convert current clients to the QCubed based packaged, I just finished writing, so right now just isn't a good time. However, now that I know what are the time expectations, I should be able to commit to participation in October/November.
I have to admit, I'm a little concerned about moving QCubed to GitHub, as I've never really managed to figure out just how it works. I guess so far it hasn't been a priority. So, if that's where it goes, I'll just have to invest some time in it. That said, if using GitHub makes it easier to get fixes and upgrades applied, maybe it's a good idea.
Except for comments about the hosting aspect, the discussion seems to have most of the other suggestions made by SP, which to me are even more important. For me, they are, more flexible integration with the JQuery UI, and documentation. The API docs are great, but there really needs to be more. Someone wrote and recently updated a QCubed Quick Start Guide, that is terrific! It would be very nice to see it promoted on the site. It would also be nice to see that guide expanded and maintained.
Anyway, I should be available for October/November. And, just for the record, I am very appreciative of all the hard and great work of the core contributors. QCubed is a great framework. BTW, there is always room for improvement, but I'm finding QCubed to perform very well.
Hello SP and the rest of the team.
For the one's that don't know me, my real name is Juan Manuel and I'm from Argentina.
I sincerely thank someone has started this post because since I know qcubed and use it I always seen as something stagnant and for that don't have much recognize in the php community.
I offer myself as a core contributor and I have been develop several interesting plugging's (multiple neested QDatagrid or TinyMCE integration +plus AJAX support) but with the current tool I have some problems to upload this code because I do not understand how it works and there is also no documentation on how to do so, this is living in my old pc =(.
I also offer to document the code and some developments since I have 4 years experience in php and 2 in jquery.
Count me in from now to the end of the year.
Qcubed is a great framework with brains really good from behind as Alex Weinstein Or Vexedpanda.
I leave my contact ...
jmirancid@gmail.com
jmirancid@hotmail.com
Feel free to add me or send me an email with the steps.
Qcubed NOT DEAD !!!
JMI
I will go ahead and create the organization on github and copy over the first commit. Any objections, let me know. I will be inviting all those that said they want to be core contributors to the organization.
Sounds good. I will try to branch with my proposed changes. Once up, I certainly will welcome feedback and help on them.
For now, there is no intention of moving to github.
The process we have for managing the changes is in place and has been working very well. I don't see the need of moving to github for now.
There is already a list of core contributors:
http://trac.qcu.be/projects/qcubed/wiki/QcubedContributors
If you want to become one, just apply, the current core contributors will vote for your acceptance into the group. We are happy to accept new members. As you have noticed the core contributors have not been very active (except for some), so new blood will be good for QCubed project.
As Alex mentionned above, I have commited myself to become more active again. Would love all the help I can get! :)
What do you think?
Hi Kmeirlaen,
I want contribute but I don't know how the qcubed actual tool works. Can you explain how i can create tickets, upload my qcontrols or how change the example page ?.
VexedPanda some time ago help me with the idea of "how this works" for upload my qcontrols but I get stuck in the middle because I made the changes in my local "_core" folder of QCubed with the QControls and Examples Page I made but I don't know how to commit this changes to the svn of qcubed. Even that I don't know if this is the way that must to be do. I think personal this is a problem for us, the "casual" contributors.
In that case my vote goes for gitHub, is more easy accomplish this kind of work.
In another hand I want to recommend, if you let me, a tool for team collaboration management and control changes. Is Open Source for PHP and is called OpenAtrium,
http://openatrium.com/
I use in my actual work together with TargetProcess and are awesome tools. I think maybe can use it and I can host the tool too if you want it. Making a public credential or create one for each core collaborator.
Just let me know team.
Regards, JMI
Have a lookt at this page:
http://trac.qcu.be/projects/qcubed/wiki/TicketWorkFlow
Also:
http://trac.qcu.be/projects/qcubed/wiki/CodingStandard
Let me know if you have further questions, we are here to help.
Hey thanks Kmeirlaen about the info links. I'm gonna finish reading this in my house and try to create a ticket with my source code of qcubed plugings. Is not seems so hard. By the way something i dont see is a ticket repository so the "casual" contributors can take tickets to work with it and solves. Any idea where are they ?
Regards, JMI
There are 3 pages of tickets right now:
http://trac.qcu.be/projects/qcubed/report/3
Everything with "in_QA" status is for core contributors to test.
Any other: go ahead and submit a patch! :)
The easiest to start of for you to just create a ticket, attach a patch based on your source code and set status to in_QA so we can test it...
Kristof
Kristof,
Are you the server owner? Can you help me with an anonymous check in to the experimental area so I can demonstrate my proposed changes? I don't think creating a patch will be very helpful in this case.
SVN is fixed! :)
It's not anonymous checkin: you need to authenticate with your QCubed username/password (same as forum).
But that should not stop you from submitting your code :)
Looking forward to have a look.
Kristof
Thanks Kristof. I checked in to experimental/qcubed21. The gist of the changes are as follows:
- Restructured the JQuery UI controls to have an extra layer. Each control now has a "Gen" and a "Base". The Gen is generated by the jquery ui doc scraper code, and the Base is designed to have core level code that makes each control play nice with QCubed.
- The JQuery UI events now go through the standard QCubed event mechanism. This allows them to be chained, and gives the control of what goes on during the initial JQuery UI object creation to the Base layer, rather than the user layer. These events also create the correct function call prototype so that javascript code can get to the 'ui' variable in the JQuery UI events.
- Draggable, Movable and Droppable are now all features you can turn on in any control. These now use the new JQuery UI mechanisms, which give them more features. The legacy code is still there to ease the transition.
- There is a new QDialog class which uses the JQuery UI dialog class in a more straightforward way, vs. the QDialogBox class, which was sort of shoehorning the JQuery UI dialog into the old paradigm. The new class makes it relatively painless to create simple dialogs that have multiple buttons that ask users for input. Yes/No dialogs and the like are now a snap.
- Added JqCheckBox and JqRadioButtons.
Look at the examples for drag and drop, movable controls, dialogs, and the JQuery UI objects example page. Be sure you have the example SQL database loaded so you can see the ajax autocomplete in action. It works well for picking items from a large list.
There are problems. Doing this work brought to light some bugs in JQuery UI. In particular, I fixed a bug in autocomplete, and checked in to JQuery UI. It is still waiting to be pulled. Not sure what is taking so long over there. Also, if you look at the drag scrolling example, you will notice the object drifts away from the mouse as you drag it and scroll. This is a long standing bug in JQuery UI that someone has yet to fix over there.
Also, when you gen your code the first time, it now creates the following in each user level model file:
public function toJsObject () {return JavaScriptHelper::toJsObject(array('value' => $this->__toString(), 'id' => $this->intId ));
}
This can be used by the autocomplete to make it easy to attach a database object to an autocomplete. However, this would not be added for people who are updating, they would have to do it themselves. Should this be pushed down into the "Gen" class for each database table instead, so that everyone will automatically get it when they update?
To Do:
- Change codegen process to generate the new controls by default rather than the old ones. Any takers?
- mikderfler, can you look at this and see how your code to return js objects could be incorporated?
Comments are welcome!
Great to see some progress.
Here is an analysis of the current situation and an outline to a vision which would bring us on the road of success again.
On the downside:
1) However the problem that I see is that qcubed is slow.
There is a huuuuuge performance problem with the way objects are generated.
Other ORMs perform 50 times faster.
The more data gets in your database the slower it gets.
The worst: It is not even possible to upgrade the server. It does not scale.
2) The autogeneration of controls I think is also no more a killerfeature as you have to change everything anyways. But realizing this every other change that we bring into the framework will be much easier to do.
On the upside:
The best thing about Qcubed is the qform part of the library that allows to build stateful forms from simple php calls and easily attach AJAX events and so on.
A very, very good achievment is that this works now with the Jquery integration. Great work folks!
Solution:
1) I think the best thing would be to switch to an alternative in the ORM.
Otherwise this will pose a thread to stay competitive with other frameworks.
But in a way that makes the system faster and get rid of some of the current OOP as well. It we make it an alternative, we won't turn people and support away. Anyway when using the new ORM, the old one must just reside in a folder untouched.
And the way to go here is definitely:
http://www.doctrine-project.org/ doctrine 2 with nosql support!!!
Also ZF is looking into this one as well. And this is the standard, this is where the community is. This is what we need to get.
I used phpactiverecords with is beautiful but it is also a dying project. Therefore we have to go to the biggest animal.
2) A field based authentification really needs to be in the core, as almost every single application that uses qcubed needs authentification.
We shall use some standard quality here. And it shall not be a plugin because then the community is not supporting it, and it only gets randomly installed.
Security can only be achieved together and having this evolving in the community will help alot.
3) If we want more marketing we have to beautify the templates. They look so 1996 at the moment. We have to change that with getting the best template out there. Make it 2011!
4) In case we are really going a bit away from the code generation. Practically there is not more needed for coding assistence than what prado does with its component composer:
http://www.pradosoft.com/demos/composer/
This is a form where you can just input what kind of fields and components you want, and it tells you the code for it. With qcubed you just generate which is easier but do not know what was exactly put where and how to adapt it.
Keep it up this is a good project. GO 3.0!!!
Hope this inspires you.
Best,
tronics
Hello,
Very good comment, I agree with a lot of things and I do feel inspired.
Some comments:
Firstly a query expression builder like in doctrine would be great as it would help with performance issues when you deal with huge data sets and more complex queries as it would allow to overcome limitations the QCubed OOP query language has. I would be happy involve myself in this as I have some ideas how to integrate this without loosing the code generation.
A class composer would be a great feature too. Agreed. That could have a lot of potential also for generating form classes.
However, we should keep (the support for) the code generation. This is a very core feature many developers like myself are relying on and enjoy to work with. A lot of QCubed applications are small business apps that work great with this feature. When I was looking for frameworks a couple of years ago I had always dreamt of such a framework.
That includes the generated forms/panels/meta-controls. I agree that you have to adopt them as e.g. list boxes are not sorted and a lot of apps need more complex forms. However, if you develop small business information systems they remove a lot of routine work and the code looks so much more elegant.
An additional thought: either the datagrid could need an overhaul or we could create a new 'slim table'. The datagrid is great but it has become a bit 'sluggish' (I have no experience with QReport though).
I have to add that one of the major advantages of this frame work is that you save a lot of time for training external/new staff. The code is very simple and easy to understand. Looking at doctrine I was not so sure. The template system also makes the communication between the designer and the developer much easier through the template system. QCubed overall is a great framework!
Just some thoughts
Cheers
Helge
EDIT:
>Looking at doctrine I was not so sure.
I think we could do a better job :-)
All,
Thanks a lot for this wonderful feedback. And I'm even more delighted to see code submition!
Now, for the progress.
If you look at the tickets for 2.0.3 (http://trac.qcu.be/projects/qcubed/query?status=infoneeded&status=in_wor...), there are a bunch of bugs that need to be fixed. I believe this is the first step in getting a stable QCubed
Next would be to have a look at the tickets for 2.1
(http://trac.qcu.be/projects/qcubed/query?status=infoneeded&status=in_wor...)
We need to decide on the features there: do we incorporate them in 2.1, or do we postpone.
We also may add tickets to 2.1, based on the work that has been done by SP and the suggested improvements highlighted in this thread.
I would suggest to organise some kind of session (chat/skype) in which we would
- assign tickets from 2.0.x to have them pass QA
- assign tickets from 2.0.x to have them fixed
- add new tickets for 2.1 (based on work done and comments above)
- go over existing tickets in 2.1 and see if we want to incorporate them in 2.1
What do you think?
Overall, I'm really really excited to see the fresh ideas. We just need some hands to spit out the code...
Kristof
For me it would be very good to make a kind of conference via skype, chat or any other. Especially for know the problems and how would be resolve and for who.
Right now I'm working in upload and making some tickets for controls and examples I develop. I started with something easy to know better this qcubed tool that lets us make changes and reported to the core contributors.
About the user comments for hdzierz and tronics.
I do not think qcubed have performance problems on massive data queries. So far had no chance to do some stress test or benchmark response compared to other ORM.
I think the main aspect of qcubed was anticipating a solution that was later used by Microsoft and it was called LINQ. I see very similar to the ORM qcubed use (today transformed into Entity Framework).
But performance on cases we have the case study that uses chess.com. A entirely application made in qcubed and I guess there have a lot of data in the database like user data, records, game tables, statistics, etc. and so far have not heard that he had performance problems, correct me if I'm wrong or if anyone is working on chess.com and want to clarify the case would be really nice.
On the other hand memory problems commenting by hdzierz on qdatagrid when handle a large number of elements are exactly the same experience I work with the datagrid to microsoft. NET Framework 2.0 (many coincidence uh ? je ;-)) and resolved by using a pseudo-memory cache was loaded once with all the database data I need and then the qdatagrid get all from the cache. The speed performance was increased on the paging and sorting by 150%.
Conclusion: I think the ORM of QCubed have enough life ahead to do a migration of it.
Kristof keep me in touch for the reunion,
Regards, JMI
I do understand that the ORM has a history and was well engineered according to the principles it follows.
However Chess.com is explained on the qcodo.com forum.
They dropped the qcodo ORM altogether because of performance issues and rely strongly only on the qform part.
So this one does not hold up as a case.
Qcubed is quite good, but to be inspired for the next level we might have to be able to question everything and be open for progress.
I do believe in doctrine because..
Check out only the release features of the 2.1 release which lists the features compared to 2.0. There is more progress in there than in any other ORM.
Link: http://www.doctrine-project.org/blog/doctrine-2-1
Also the community is very robust.
And it can do MySQL and No SQL. Or any other via the Abstraction Layer which is optional (as these always compromise a little performance).
//
However I totally understand in case the community here decides on the current way to do things.
We could improve performance by making for example an alternative QListBox by removing object instantiation for every single entry in a dropdown field. The ->SelectedValue might have only the ID instead of the complete row object (with all field objects) which you can use to instantiate yourself. This could be a startingpoint: https://github.com/qcodo/qcodo/commit/0a9359ce183f9c98173461d01a5720653d...
This would improve performance of forms a lot and is something that I heavily encourage. By having a different name for this qform we would stay backwardscompatible (in case we need it).
All the best,
tronics
Qcubed with a Doctrine 2 ORM? I am salivating at the thought. This is no small task, however. I tried de-coupling the ORM to use it with Zend Framework and it is a task for sure. It would be nice to see the ORM isolated and allow choosing the ORM of preference. This would probably require different code gen templates depending on the ORM, but it is just a thought.
I already achieved ORM integration with the wonderful phpactiverecords ORM PHP5.3 only. Search the tickettracker.
This one works well. Even side by side with Qcodo ORM when giving a different namespaceprefix which is something I have requested from the creator to explain with concrete examples in their forum.
You can generate and then just replace the ORM part of the things taking too much performacne when working with this one.
http://www.phpactiverecord.org/
--
Talking again of Doctrine ORM (btw. based on hibernate which is the ORM of choice for Java).. it can do codegeneration out of the graphical model!
You can model the database with MySQL Workbench desktop program.
And then deploy these changes to you mysql server and generate doctrine models on the fly. You can even have versioning in the database changes.
http://www.mysql.de/downloads/workbench/
Workbench is a heavily updated and producers supported program. Also it is free of charge.
Graphical modelling is IMHO more appealing than fiddling in phpmyadmin.
Cheers,
tronics
>code gen templates
This is where I'm not so sure.. probably leave the ones we have now in there and improve them on the Qform part.
But adding a high performance ORM of choice alongside. And do the models with generating from the graphical modeling.
Changing the templates might not even work, as doctrine does some things differently.
--
For the authentication that I mentioned as really nice to have above, I have developed a nice startingpoint with my partner and released it as a plugin already. But community power is needed to make this more mature and have the field based authentication (which already works) now done without different signature on the template side.
Best,
tronics
Maybe I am picky, but I wouldn't want to have 2 different ORM's on my app. I would prefer to get rid of one and use a different one.
>Maybe I am picky, but I wouldn't want to have 2 different ORM's on my app. I >would prefer to get rid of one and use a different one.
I just meant to say that I was able to codegenerate and still use the other ORM, so compatibility is there.
Sure it is not very convenient to have two ORMs used at the same time.
We could ship both ORMs and the programmer can decide to delete the folder containing the unused ORM.
I just installed the latest version of Qcubed along with several plugins and played around for a while.
The community here built a really nice system. It is most valuable and looking fine!! Great work. Really important to make that statement. I appreciate all the effort.
The suggestions were made to get people inspired for the 3.0 vision.
If we improve Jquery and hunt some bugs for 2.0.3 a lot is achieved and having another release after 6 month will definitely be a good sign for all people involved.
Could you explain a little more what you mean? Are you specifically talking about code generation? When you say that the more data gets in your database, the slower it gets, doesn't quite make sense, since it is only reading the table structure, and not the data records. Do you mean the more tables and columns?
If you are talking about situations where a list box is created, yeah, that will happen. A list box needs all of its items listed in the form to display. If you want a different behavior, you have to change it to some other kind of object, either a data grid with links to select each item, or the new jquery autocomplete object with ajax that I put into the experimental section could work. There currently is no design goal for the framework to automatically detect that a particular kind of object would be inappropriate for a particular situation based on the size of the database.
The only thing inherently slow about the framework is that it returns all the columns for each record it searches for. The framework is designed so that you can hand code sql statements in the situations where this is a problem.
Changes to the framework should be guided by overall design goals, not just neat technology. For those of you who like Doctrine, could you articulate how it fits in with the design goals, or articulate how the design goals should change to fit it in, and what in particular you like (as in, is it just DQL you want, or are there other things?)
As I understand it, the current design goals of QCubed are:
Did I forget anything? Would you change or add anything?
Thank you for this post.
I'm fine with the goals of QCubed they were well thought out.
I would add that we should try to ship best practice right out of the generated code.
>If you are talking about situations where a list box is created, yeah, that will happen.
You are right, it is actually the generated qforms that do not scale (sorry that I put so much stress on the ORM which was actually not right).
There is too much object instantiation in the way that e.g. QListBox is set up. Therefore I would go as far as saying the current implementation of QListBox is not really feasible.
Developers build their application around the suggested code and basically all of their apps get really slow over time.
The ORM itself also has a certain overhead. A normal object generation takes only 33% of what Qcubed needs but still scales. (You can read about it here: http://qcu.be/content/qlistbox-holds-all-objects-making-it-slow-certain-...)
But this is something that I could live with.
Do you think it is possible to have a second QListBox that does not use the objects when codegenerating?
But instead only uses a customquery to fill id, value pairs and then uses the ORM to load he object only when it is necessary (in case there is a SelectedValue already) later.
--
Regarding doctrine, it does not fit the current goals of QCubed.
There are advantages from this new tool. Eg. constant development that we do not have to take care about. Good performance, possiblity to use NoSQL.
However having a plugin that let us use this side by side with QCubed would be ok as well. And serve as a marketing tool of course.
--
I think we could benefit if we reinvent the QListBox.
I have found a thought out contribution here: http://www.qcodo.com/issues/view.php/42 It might be a good startpoint.
Best,
tronics
Sure, you could pretty easily code that. But eventually you will suffer the same problem. When you get 1000 items in the list, the page will be slow to load, and worse, it will be really hard to use; it will take forever for the user to actually pick the right item from the list. You need to somehow put a limit on what you show. This has come up in these forums before.
The only real answer I have seen is using some kind of field that filters the list as you type, or does a search after you type. The filtered datagrids are one attempt at this. I have coded similar kinds of filters that use one field to filter for that value in many different columns simultaneously and then display a datagrid showing the various options that match.
Another way of selecting from a filtered list is using autocomplete. Please check out the SVN respository and see what I posted in the /experimental/qcubed21 directory. In particular, on the demo page that shows all the JQuery UI objects, there is an Ajax autocomplete object that works really well for presenting a large list of items to choose from in a way that allows the page to load immediately, and then loads a subset of the list items using ajax as the user types. While it still loads the full objects at the server side, it doesn't have to load the entire table. At the client side it will only see ids and names, and its reasonably responsive. Better yet, its easy to implement from the programmer side, and easy to use from the user perspective.
Perhaps it makes sense moving forward to change the templates so that instead of drop down lists, it presents an autocomplete list? From a user perspective, I think that would be annoying. A standard list works really well for choosing from up to about 50 items. I think choosing from a list is one of those tasks that the programmer has to carefully think about how to implement in each situation. I don't think its something QCubed will be able to get right every time by default.
Ok, thank you for your analysis. You are right in many points.
Do you please have an example where I can see how the QDataGrid is configured to replace the QListBox?
However let's continue the discussion for a bit to see if there is a way to improve it. I think I found 2 relatively easy points we can improve.
>Perhaps it makes sense moving forward to change the templates
>so that instead of drop down lists, it presents an autocomplete list?
>From a user perspective, I think that would be annoying.
You are right. In many cases autocomplete is unfortunately also a bit annoying for the user because he has to type something but came to this field because he wanted to select something.
Also I think we should not automatically change the interface (was suggested earlier) when there are more entries then a specific threshold. This might lead to confusion.
----
Here are a few arguments why we shall somehow find a better solution:
- If you have 8 QListBoxes with 80 entries the performance is a problem (although one can argue the usability is still ok, as for each letter in the alphabet there are still not too many names in the list). Also when the table is large, like eg. an table holding addresses and comments in say 20 fields then the performance will decrease with each entry.
- Had a project where the QListBox hold 300 entries but there is a filtering JS above it. So people can scroll on the Listbox and read the actual values sorted, OR they can search for concrete contents.
That would work so much better without the overhead of generating objects for each of the entries.
- Some CSS/JQuery plugins also like to use a select as the basis, eg. a slider.
And when we have a large interface with say 50 formelements where 40 are based on listboxes the page will perform very bad. However taking just the HTML of that page and delivering it compressed is really very fast, the size of the page is absolutely no problem, only the overhead is.
----
2 Suggestions:
1) There should be one example page that shows the 3 options:
QListBox, QAutoComplete, QDataGrid (configured as a ListBox Alternative)
This would make it easy for people to
a) see the implementation next by each other, so they can switch it easily
b) people do not have to be inventive to redefine a QDataGrid as a QListBox replacement, I never came to this idea. Especially because I think that QDataGrid is also performing quite slow and users need many clicks to get from a to z (not every user is intelligent enough to use the filtering first).
c) read the pros and cons
This makes it really transparent for the user and will help a lot of people starting out, as well as people that did join the project before filtered datagrids were working, or more recently before autocomplete worked.
2) We write a small wrapper for QAutoComplete, QDataGrid (configured as a ListBox Alternative) that allows to be used just like a QListBox.
So that people can just replace their QListBox with a eg. 'QListBoxAutoComplete', that is taking the same parameters and is already configured to act as a QListbox replacement (small, filtering enabled, etc.).
Best,
tronics
Hello there,
I am not a great expert at this framework, or PHP, or programming or anything else. But when i set out to build my app, I browsed through everything and ended up with Qcodo, because Wikipedia has a mention! I went through their tutorials and they were great! Then I landed up on QCodo, and then on QC (well, I am lazy enough to shorten QCubed to QC) examples site and realized that those called symphony, Yii, CakePHP and CodeIgniter were KIDS. I got everything I wanted - AJAX, Form Handling, Event Driven Programming, Database handling and what not with QC.
My app might want real high performance someday. But I wanted to start off with something simple. Well, to be frank I dont find any SQL DB (be it MySQl, PG, SQL SERVER or _bloated_ Oracle) to be 'fast'. Moreover, things need to be more dynamic than what is being offered. And I do find Qcubed is, well, slow! But then, the fundamental thing: get it working first, optimize later comes to mind. I might want to shift to a NoSQL solution but then, its all because I am able to see that creating what I wanted is a lot easier task with QC than without it.
Hosting Code
Well, I am ready to help. But I am real short on cash. All I have is a shared hosting account and I am ready to offer the space. You can ask me to mirror the downloads (they say I have unlimited bandwidth, so).
Documentation
As another help: I get some money from those Google Ads put up on my site (don't worry, I am not going to ask you people click it) and I dont get much because there really are very few things I write about. I mostly am busy with code. But I really want to help QC. I see its dying and lack of documentation, examples, tips etc only gives more poison to the framework.
I would want to write some tips and a few other things (like I could write on 'Why Code Generation is Great?' or 'How to remove the AJAX error causes due to session expiration?' or may be on 'How to create your own Login classes based on QC?' and Stuff. I have a simple request: I would happily allow you to copy the stuff from the pages I write and add to documentation, but please, if possible, do mention my contribution in the documentation page as a link to my pages. I have so much to write about QCubed but I cannot really participate in the Documentation things actively due to the time constraints. But seeing this thread had made me think about 'What have I really returned to QC for giving me so much ease?'; but then, I am really short on resources. May be some traffic on my site would help me, as well as QC.
I would submit a few links on forum after I have written them. Also, how about laying out the plans for the 'Documentation'. Like what chapters, sub chapters should be there, in what order. I would like to pour in whatever possible. But let's have a Documentation plan. And well, I do apologize if I sound real mean about mentioning the contribution.
Features
I did write some classes for my personal use on the app. I would have loved to add those to the QC core f/w but I haven't been able to figure out everything about how to do it. Moreover, submitting patches is something I have never done and it does not really turn me on. But, I must say there have been a few occassions when I wanted that what I have written could have been a part of QC.
I would want to submit those files / classes I create to help myself and someone who is really into the 'core' framework stuff could think about integrating it? Does it sound good enough?
Reagrds
SP wrote:
There is a ticket with a patch for that:
http://trac.qcu.be/projects/qcubed/ticket/79
It would be great if people who are interested in this feature (only selecting specific columns with QQuery), could give it a try.
-Vartan
Hello,
This patch is very interesting, could make a big difference and I would love to test it.
Tried to put the patch manually, but there is still a little problem.
However I found a minor problem with fetching from SVN:
http://imageshack.us/photo/my-images/713/unbenanntby.png/
http://imageshack.us/photo/my-images/715/unbenann2.png/
This plugin asks for username and password and does not let me checkout any further, so I'm kept from fetching the qcubed source to apply the patch.
Best,
tronics.
Does QC have a feature to use memcached for storing data into cache?
I think that this is not the job of qcubed. We use memcached a lot for a distributed system using java for the db-system and qcubed as a front-end (with xml-rpc) relying on memcached for caching. By using global "manager" classes we access the database(remote system)
You don't need any qcubed specific things to store data to memcache
(if you are talking about form states ... ==> high memory-usage -> lose data --> bad idea)
what kind of data do you want to store into memcache?
Just ask if you have any specific questions about memcached usage :)
BUT I think this is the wrong thread. So open a new one for further questions ;)
Depends what you mean. I think you cannot cache form objects or controls, but the generated ORM objects have QueryArrayCached functions that use the object QCache to cache the query result.
Hmmmm.....i think yes it is not a job of qcubed to do memchaching as such.
I wanted to cache query result objects and i wanted to make sure that cacge is not generated separately for each user but a global cache for my front page. :)
Moreover i am wanting to use mongodb with qcubed. Can some expert guide me over this? I hioe that is not a bad idea. Or shud i open another thread for this? :)
Well ... you are right! Query caching by using memcached as a backend would be great :)
But I think qcubed has no such feature for now. I remember a friend of mine implementing such a feature but the modifications to the code-generator were rather heavy.
Well, I wasn't actually thinking about implementing it into the Code Generator itself. I actually thought about creating a class which could be reused for caching.
QCache uses files as cache so I would not think that it would be extended. But, there are certain queries which when cached can give real performance boost (comparing to other queries in a system). For example, on a site, the front page is usually the most heavily browsed page. Other than that, the elements such as the sidebar lists (which may be produced using Queries at times as well) are also heavily reused. Using CodeGen essentially would push memcached operation everywhere and that might not be the best way to use it.
So, if there could be a basic piece of code (i mean a class) which could benefit from memcached without actually being pushed into the codegen, then it would be simply great! Is there some work already going on?
This really doesn't seem like it would be real difficult. I threw this together looking at the existing QCache class and the codegen template qcubed_query_methods.tpl. Purely a concept, not even sure if the code would work, but it could be a starting point for someone.
QMemcache
<?php
class QMemcache extends Memcache{
const MEMCACHE_SERVER = '127.0.0.1';
const MEMCACHE_PORT = 11211;
const MEMCACHE_EXPIRY = 100;
public function __construct(){
$this->connect(QMemcache::MEMCACHE_SERVER, QMemcache::MEMCACHE_PORT) or
die ("Unable to connect to Memcache server");
}
}
?>
Codegen addition to qcubed_query_methods.tpl
<?php
public static function QueryArrayMemcached(QQCondition $objConditions, $objOptionalClauses = null, $mixParameterArray = null, $blnForceUpdate = false) {
// Get the Database Object for this Class
$objDatabase = <%= $objTable->ClassName %>::GetDatabase();
$strQuery = <%= $objTable->ClassName %>::BuildQueryStatement($objQueryBuilder, $objConditions, $objOptionalClauses, $mixParameterArray, false);
$strKey = 'qquery/<%= strtolower($objTable->ClassName) %>';
$objCache = new QMemcache();
$cacheData = $objCache->get($strKey);
if (!$cacheData || $blnForceUpdate) {
$objDbResult = $objQueryBuilder->Database->Query($strQuery);
$arrResult = <%= $objTable->ClassName %>::InstantiateDbResult($objDbResult, $objQueryBuilder->ExpandAsArrayNodes, $objQueryBuilder->ColumnAliasArray);
$objCache->set($strKey, serialize($arrResult), false, QMemcache::MEMCACHE_EXPIRY);
} else {
$arrResult = unserialize($cacheData);
}
return $arrResult;
}
?>
Shoudln't we start a new thread for this?
Cheers
Helge
+1 to hdzierz.
I would love to have something in there. :)
Someone should. And I won't coz I think it should be done by some senior member here.
Vaibhav
Please do start a new thread on this :)
BTW, there are no senior members here. This is a community. No babysitting. Everyone is equal. Everyone is encouraged to contribute. We want your help!!
Started new thread for memcached discussion
http://qcu.be/content/using-memcached